vvvv-Message
pack
Credits: devvvs, Json.NET, MsgPack, vux, Timing from tmp
about
The vvvv-Message pack has been in open development for four years. In that time it grew alongside free and commercial projects involving the likes such as Projection Mappings, Device Bindings, UI Widget Handling, On-Site Data Plumbing, Web Parsing, Boygroup Replacement, and not least, Bus Driven Patch Architecture.
The pack is exposing the SharpMessage functionality to vvvv in a fully spreadable way.
Install
Alternatively, the latest releases and prereleases will always be at the GitHub Release page, just create a directory *packs/vvvv-Message* and unpack to there.
Getting Started
Use F1. Look at the help patches, if you are in doubt, because there is a lot of information in them. The shortest route to try out Message, however, is the 101 girlpower.
Anyway, first two nodes advised to check out are ConfigKeep and Split.
If you feel comfortable with splitting and maybe even editing, understand that a Message can become stateful (i.e. across time, not just a single vvvv frame) when kept inside a Keep.
Mission
The original idea of this pack sounds convoluted: make plugins with easily typeable io pins and use that to retain order in bigger patches and saving lots and lots of links by simply packing all kinds of attributes in a single meaningful object and handling that object in a very easy and data flowing manner. But it is plain: teach vvvv how to write layouted letters and fill them into marked envelopes, meant to be given from hand to hand.
Goes without saying, but every hand in this metaphor is a of course a node. And there are no letters or envelopes of any paper or other material kind, but only smart C# Messages, trademarked with a proper Topic and automatic localized timestamping.
Use nodes to create, read and write, reshape, analyze, search and sift Messages, and, probably most useful, keep them and use them as stateful objects that can be manipulated anywhere else in your patches' folds.
Message is meant to be open, it strives to connect with other relevant protocols.
Features
42 vvvv nodes total: Don't let the number fool you, most of the time you will not need more than a dozen of them, but this is a vetted pro pack. So you get the full suite to freely evaluate and even use it in non-commercial work, as seems fair.
- At first glance, it might sound the most boring of all, that you can tightly define the structures of your Messages, both per-node (thankfully in a custom vvvv window) and for many nodes simultaneously across entire applications. This feature is called Formular (yay, more Kafka to the vvvv)!
- Industry standard serialisation with both Json.NET and binary MsgPack are available to help with persisting, streaming or even distributing data among vvvv instances - local and networked. Additionally it helps to establish api bindings with good ole' OSC.
- It comes with girlpower examples showing how to connect to TouchOSC, Ableton, Reaktor, XTouch midi interface and Duration, just to showcase its broad versatility and making all these things easily accessible through Message manipulation.
- For reliable ethernet comm with Message check out ZeroMQ.
- Extending traditional data-flow, the pack allows down-stream edits (i.e. Message is conciously mutable). This is big, because it helps any project, where you need read-write access to your data across patches without having to resort to framedelays. The pack's change management is just the icing to the fact.
Oh, you can also put DX11 resources into a Message. Or a spread of Messages in a Message. However deep you'd like to go. But that is totally up to you.
License
This is released with Creative Commons BY-NC-SA 4.0, so no commercial use!
If you need a license other than that contact us at license@intolight.de
Consider that intolight rates start at €42 per vvvv license, and compare that to the productivity you gained.
download
anonymous user login
Shoutbox
~1d ago
~1d ago
~9d ago
~11d ago
~12d ago
~16d ago
~16d ago
~23d ago
~30d ago
~30d ago
Full code can be found athttps://github.com/velcrome/vvvv-Message
I like tidy patches
cool!
hey it in theory it makes my life easier, thanks for sharing :)
testing it tomorrow.
Okey super cool!
hi thanks looks like very nice concept ;D
In the Message split i get the maximum spread count of any input value instead of diferent spreads count depending of if types , is this the normal behaviour ?
cheers
the expected behaviour relies on correct binsizing.
this might sound weird at first, but if you have different spreadcounts in different messages, you will need to spread the bin size.
hi cool yes all fine as long as i specify right binsize per type , thanks
I added an example for Duration
Perfect timing for the Duration-example. Thanks vel!
I've got this error by opening "Message.csproj":
Seems like the TypeIdentiy.cs is missing.
thanks for the hint. forgot the file. please download again.
and as a free bonus, i added my sample project for duration too
WhoopWhoop!
All systems are up and running now.
Thx.
Works out of the box with TouchOSC Layout "Simple" on Android.
Nice work Thnx :) Readme.md?
> Readme.md?
Markdown file. easily readable plaintext with formatting for nice rendering, too. Although I have to admit I've never seen a markdown file with html-style tags, this kinda defeats the purpose ^^ (e.g.
vs.
uo, ok. didn't know. <code> created these wonderful node-like boxes around the words. seehttps://github.com/velcrome/vvvv-Message/blob/master/README.md
these are also created with backticks (`) ;-)
also, bold text can be had with double stars, **like this**, instead of <b> tags.
github adds some additional nice things, see https://help.github.com/articles/github-flavored-markdown
thanks for shoving my nose into it. was well worth the trip into Markdown. for exercise i updated the md.
fun fact: some inline html is explicitly viable in Markdown. but i agree, it does feel more backward oriented
Cheers,
pretty handy contribution, using it a lot.
But when using, i was thinking about the necessity of the connections at all.
Why not declare Message to the new S+R Nodes...!?
Message(Join) as a generic Datatype, that can handle everything and Message(Split) just as an reciever. So instead of the "Message-Input" of Message(Split), there could be just the Adress of some Message.
And with Message-Type there is a way of sorting adresses.
For Example when Message(Join) is type Event and Message(Split) also, Message split would only show up Messages of Type Event.
Would be The End of all the S+R hassle and patching would be much more <3!
Just some thoughts...
Ahoi
d0t
add to addonpack ?
hey chris,
i'd rather not add it to the common addonpack. tbh i'd like to develop a solid pack out of it, because i consider it most useful only for advanced application development. most work is done in that direction, as you can see here
alright, i love this plugin and use it quite often. i wouldnt mind it being in the addonpack at some point
Hey Velcrome,
i´ve tried to install it without success, red nodes everywhere even when i drop the nodes manually,
The same problem with b32 and b31.2.
Any tip for a correct installation?
thanks in advance
@lasal unfortunately the latest zips are incomplete.
You can clone the stuff from github and compile it yourself. Velcrome did a really nice job (everything is in order and dependencies are handled via nuget) so it should just work.
Thanks Bjoern,
i´ll try it, or find someone with a working compiled version.
here b31.2 working, b32-not
i just downloaded a fresh beta32_x64 and the pack, it works fine for me. same for the x86 version.
could you please start ^code:Demo (Message).v4p^ and share the tty renderers output, so I can look into your problem?
Hey Velcrome,
thanks man,
In the 64 bit version there are some red nodes, buffer, ringbuffer,
also Cons (message) is missing in the duration example, and Getattribute in the TouchOSC example.
TTY:
In the 32 bit version everything is red.
TTY:
The first impression with the 64 bit and the duration demo is, waw, great job.
I found and fixed some errors and did some more restructuring.
Also you can use the Message datatype now as a nuget dependency, simply search for VVVV.Packs.Message on nuget.org
Hey,
sorry for the delay,
it works basically in the 2 versions, but ringbuffer, buffer and cons are still red,
thanks
@velcrome, you are aware of generic-nodes-project , right?
thanks for the feedback. and yes i make good use of the generic nodes from the sdk. they are truely awesome, can't wait till nuget can be used from within our toolkit.
As you can see from my short-witted comments in the source i decided against keeping ringbuffer, buffer and cons. if i were to elaborate I'd say Zip does everything Cons does. I did replace it when cons did not behave as expected some alphas and betas ago. Even though these problems are probably fixed, I did not feel the need for a Cons since.
As for Buffer and Ringbuffer - I could not find use for them, but you can implement them yourself with 2 lines of code. I did not omit them because of laziness, but by choice of simplicity. I have not had a use case yet that couldn't be done with Store (Message).
If you have a use case, add a reference to %vvvv\lib\core\VVVV.Nodes.Generic.dll in the project explorer and a using VVVV.Nodes.Generic
hello velocrome, thanks for the great and usefull plugin.
is it working in v32 in the 32bit version ?
for me it is not. ill provide tty tomorrow
nevermind i used an old version + didnt use it as a package..
sorry
hey lala, nice to hear you like it. and that everything works as expected.
trying to get latest version running but red nodes everywhere. All the demo files reference csproj files suggesting dynamic plugins but there are no csproj files. What's going on here?
thanks
which version did you try, x64 or x86? the one that is for download on the top of the page?
if you used the latest version from github you will have to compile in a different ide than vvvv, because as of yet vvvv does not support native nuget management.
anyway, thanks for hint with the csproj. if the pack is properly installed in /packs, it will automatically replace the nodes by their compiled counterpart in VVVV.Nodes.Messaging.dll, but will not reflect this autoreplace after saving the v4p, this needs to be done manually.
I am having some trouble in beta33.3 both in x86 and x64 I only get ø out of the nodes in the help patches.
mmh, cannot confirm yet. which help patches are you talking about exactly?
Currently i am in the process of revamping the Message contribution. I reached a alpha state, where some feedback would be most useful.
x64
x86
101.v4p is probably the best way to start.
@velcrome: ooh! bunch of stuffs to play with! (also i've sent you an email a long time ago check that :P)
in 101.v4p "Time" node is missing :(
hey microdee, sorry for not answering yet. let's chat some time.
btw, the missing node is in tmp's vvvv-Time.
damn why did that go under my radar? seems superuseful
getting closer...
new alpha:
x64
x86
One question about performance:
My message would have the complexity of let's say 5 spreads of 8 floats each and 5 spreads of 4 bools each.
If a MIDI message changes a single value, in practise one of the spreads has to be sent and parsed for each MIDI message coming in, right? Unless i split the message into having 60 individual members.
I can't imagine that latency will stay down either way?
In comparison to Eksposing, isn't this a lot more string parsing behind the curtain?
Disclaimer: I'd be using about six different instances of such a message receiver with individual addresses.
For one, there is no string parsing going on behind the curtain. Remember, you are directly manipulating .net types, not some obscure comma-separated string representation of them.
Your case seems like a walk in the park for Message.
However, performance is a big topic for Message, so it would be awesome to get more feedback if you hit certain limits. I admit I have postponed most possible optimisation for a time when usage tells me what to optimize.
Right now I have a application with some 70+ widgets with a whole range of attributes like size, pos, color, Command when hit, type, source, etc. As long as they are not edited, there is no perf hit at all, because Message plugins will only do work when change has occurred (from upstream, and for Keep plugins, even downstream, but deferred by one frame).
But if change occurs, all the Message plugins will evaluate which might have a negative impact on framerate.
Note to self: Especially the deferred change detection from the Keeps seem to be too massive to be on by default.
edit: for your case, you should start with defining a Formular for your type, feed some Defaults into a Hold, and then use Edit nodes downstream. Feel free to post a patch for more help.
the S+H message node is a framerate killer apparently.
This new version is so fresh, some critical features are only working with the current alpha.
It now comes with a quick "Formular" definition to type your stuff across your vvvv application, with an advanced concept to hold Messages that I coined "Keep" and some nifty pin-management to detect actual changes and only then get active.
Dive in quickly with the 101 course in the girlpower directory. For more detailed information book one of the last few spots
As always, feedback is welcome, and so is a flattr.
velcrome: Couldn't be with you at "vobjects and messages". Is there a recording of it?
no, not to my knowledge.
the next best thing would be to download a current alpha and try the 101.v4p in /girlpower.
Finally getting around to exploring this, but running into some problems and tons of red nodes in patches using a previous version and the current girlpower examples.
First, the file above:
vvvv-Message_Release_x86.zip - 17.04.1514:56 UTC
seems to be packed wrong, as it seems to have folders recursed such as nodes/core and nodes/nodes, and a different version of VVVV.Packs.Messaging.dll in the two core folders. I got it to work by taking the bottom-most nodes and using those, but not sure I got the right version of VVVV.Packs.Messaging.dll.
Then the girlpower examples, such as Durations gives red nodes for Message and AsMessage; looks like those have been renamed
Looks very cool, anxious to experiment with it, thanks a ton!
confirm, will look for a solution. i usually only need the x64 variants, so x86 doesn't get the same attention.
on a side note, the github version is much better right now, but requires compiling it.
edit: done. there has been some work under the hood since 2.0, because vvvv-Message was being used for in-house projects, so quite a few quirks surfaced and were consequently improved. have fun
Fabulous! Looks good - thank you sir!
It's been over a year since the last official update here on this site.
As some of you might know, there has been a lot of development on Message, usually freely accessible on github, but never here. To give everybody a chance to do some proper alpha testing (and general tinkering) without the need to compile yourself, I want to leave the release candidate for the next official release.
Download from our owncloud, and unpack both zips to the /packs folder of a current alpha vvvv install.
You will need to install dx11-vvvv as well.
Most recent nuget for the core (if you desire) is here.
If all goes well, this will be the release targeting vvvv beta35.
good stuff ! will you do a workshop about this at next node festival ? would be cool..
Yes, I would like that, too. Even though I still don't know how to properly explain the pack, because its purpose is pretty high-level. And exotic "unfrozen object oriented data streams" are hard to picture, much harder than, say, imagepack or dx11.
One thing I learned from the last workshop is, that this pack offers solutions to problems, that users will only encounter after a few years of vvvv practice. So this time it should definitely cater to advanced users from the start, where I define Advanced as "has abandoned at least one promising project of her/his own making out of disgust about the unmaintainable patching".
Last two times I simply glided along the girlpowers, but that's not saying too much about the conceptual side of using vvvv-Message, like where to start and what to consider.
So requests for live-patching relatable use-cases are very welcome!
you probably made message to solve your own vvvv patching problems. it would be interesting to hear about them and how message solved them. i wonder about the ratio between "elegant solution" and additional node noise caused by message.
I would be a pretty lousy programmer, if vvvv-Message just solved my problems, after more than 3 years of on-off development ;)
Roughly, I spend that time:
40% node finetuning for rapid patching
15% getting to grips with c# (yes, this has been a tremendous learning experience)
20% solid, independent core
20% project infrastructure, documentation
If this were for my own problems only, I could have cut down on dev time. I might even have sticked to poco, just because I can.
But I believe, that this object oriented flow approach can help greatly with having some actual architecture in your projects. It can enhance productivity with vvvv at the top end, when stuff gets really complicated for a patching environment.
That said, of course a lot of intolight project problems, or rather their solutions found their way into the pack, one way or the other. But that's natural, I guess.
In a nutshell, every serious patcher will eventually confront the problem of having a more "complex state", that needs to be made available to multiple modules, maybe even across multiple instances and machines. The standard way is to calculate that state somewhere singular, and send it (usually through multiple links, or even multiple S+R, or infamous hand-crafted serialisation modules) there, and put back together the pieces you need.
With message, you can put your state data into a single object (or multiple, however you like), and noodle that thing around, through one link. You can send it over network and serialize it to disk in a standardized manner.
You can partition the state-calculation into more than one module, that can act independent of, or interlinked with each other.
The elegance is plain: Everything is tangible in patch, and everything is geared for fast patching. The revolution is, that changes to a Message can affect Keeps that are connected only upstream, without having to use any FrameDelay (which I am sure gregsn would snarl at, hehe).
There is an absolute minimum of stateful nodes (only 4.5 of them right now). The rest is stateless in regards to flow data, which makes the kit amazingly straightforward to use, as long as you keep in mind, that any change to a Message will be so across your entire patch.
The nodes themselves are usually a tad more high-level than you would expect, so you can do much more with less.
The nodes of vvvv-Message know about the structure of the Message data (unlike a Zip and Unzip), and this knowledge is heavily utilized, so you don't have to constantly worry about it.
Here you see, how using Message will clear out half a dozen nodes, and make your patch more readable, while actually gaining possibilities (like making all data available to other modules, that can split out only the Fields of interest, or even edit it).
I guess you'll just have to try it yourself, and find some patterns (usually around the choice of a Keep), that suits your needs. That said, I am sure, this pack will provide you with moments of "Shit, how can this be so easy. This was soo hard all these years!".
Because towards making those moments happen I spend the remaining 5%.
Yes, totally elegant solution.
But when not using networking/osc and I only needed some custom datatypes to save myself from wiring loads of links through many levels of patches, woei's Struct seemed much simpler with less node overhead.
I guess Message offers way more convenience and features that may be needed at times, but I sadly just didn't have the patience to wrap my mind around of it works, as it seemed to need way more nodes for a simple setup (Keep volatile Messages vs. permanent Structs).
So, looking forward to learn more about this, pretty sure it solves other problems I have at times.
In the past, vvvv-Message lacked documentation, and I can understand anyone not investing the time with all these nodes, all doing very strange things here.
Because for basic scenarios, you might have to first learn them all? And to add to the bizarr: Just to read this, you'd have to scroll through endless lists of exceptions, that these nodes are producing?
But again, everything you know is wrong.
Start with a ConfigKeep node, and go from there with a Split, an Edit maybe. Those three nodes alone can make a tremendous difference in your patching experience. Read help patches, and learn at your own pace.
This is a major release here, and it's got half a year extra to be polished, thanks to the late release of beta35. So the suite is tested, prod ready and documented- also in code and patch.
Hello velcrome!
I have for the last few weeks been getting to grips with your message pack and really loving it. I am using it for an elaborate preset loading and saving mechanism of about 140 different parameters, each with about 10 attributes.
To be able to load older presets even if I add new things, I need a way to edit the existing messages and overwrite the attributes of all the messages with existing topics. Edit(Message) does the trick, but I cannot specify topics there, so it just edits the values in the order they came in. I need something like Edit, but being able to specify which topics to change.
Say I set up using ConfigKeep 3 messages:
topic:Speed,
property:Speed,
value:1
topic:Lightness,
property:Lightness,
value:0.5
topic:Saturation,
property:Saturation,
value:0.1
Now I have a preset I want to load, which contains those messages as well. I can just read the preset (from .json file), split and use edit with Force to inject the new values.
But how can I edit those 3 messages if the preset I am loading only contains 2 of those and in a different order? I assumed I could just use ConfigKeep again but Force it, but that doesn't seem to work as expected - it only changes the messages coming out of ConfigKeep, but not the messages of the same topic and formular globally!? To be able to get to the topic I actually have another field called property, which uses the same string as the topic.
I need something like EditByTopic, where I can input a bunch of messages like Edit, but then have an input for Topic and any of the fields I choose like ConfigKeep and it sets the values for the messages with the existing topics.
I hope this makes some sense. Best, Armin.
Sift is your friend, if you want to filter by Topic (which often happens!). This would be between your ConfigKeep and the (forced) Edit.
Alternatively you can Split or Read all your Messages, and check the binsize of the attributes, to see if a field is not initialized, and use that to Select only the incomplete Messages to fix them with Edit. Or you use it to spread the Update pin of the Edit, to make sure you only update incomplete Messages.
There are really a few ways to do what you want, even without coding a new Edit node. The most brutal is Inject, but even ContainsField might be worth a look.
On a sidenode: currently there is no such thing as a "Field Order", so it does not matter, in what order they are in the json.
Hope that helps :)
Edit: @seltzdesign chatted me up directly, and explained his problem in depth. The easiest solution turned out to combine an Inject with a PruneBut.
If you want to know why, leave a like here and participate in Node17
Message looks broken in vvvv beta35.5.
Even in the help patches all nodes are red and without inputs.
Excerpt from the log file:
@mfo
Cannot confirm on a fresh beta35.5_x64 with vvvv-Message, installed with vpm. Unfortunately the log you provided does not show any hint of the reason this happens to you.
Can you please start a new thread (or a github issue, as you wish) with a link to a full startup log? Please also leave details, like your pc specs, and do you use x64 too? Which dx11-vvvv (or any other packs) do you use?
This pack has been successfully updated to beta35.7 and dx11-vvvv 0.1.44
@mfo (and velcrome)
i think i had the same problem: when I downloaded the pack from github (as zip), for some reasons the dll's in the folder >nodes>plugins are not included, thus the nodes were not there at all.
It worked with vpm in the end.
@aivenhoe is probably right, I've downloaded the latest zip from GitHub (compiled against beta35.8) and it is working nicely.
Great work @velcrome , this is a beautiful tool!
EDIT: Spoke too soon. Create and Split aren't working for me with new Formulars. Old patches do work, but changing or making new formulars doesn't.
@mfo can you share the patch? The Formular node turns red, if it cannot parse your definition and will reject the update bang. That can admittedly be confusing, but I never had a case, where this "didn't work" for a valid definition.
Also, the vpm script downloads and unzips the very same zips from the github releases, so missing dlls seem unlikely (maybe try with 7zip.exe?!)
Dunno how to upload files here. https://we.tl/FDRHatGcKS
There isn't much in the file anyways. No red nodes, no exceptions thrown. A simple fresh Formular, just a String Foo. After updating neither the Create nor the Split node show a pin for Foo.
ah, now i understand.
After you changed your Formular, simply click right on the Create (if you haven't yet) and check the little box in front of your changed Field. Same for Split.
Voila
Again and again, messages comes to the rescue. After quite a learning curve I think I got the basics now. Storing and loading presets in json format with around 250 different settings and 10 parameters each works nicely. Also use it in situations where you would need multiple send nodes and its working really well.
thanks for taking the time to make and improve messages.