I learned about Node-RED this week. That is already a surprise to me. The product had its first commit in September 2013 by the hand of Nicholas O'Leary. I would have expected to bump into it sooner, not over 10 years later. At that time, I was using (and still am) Pentaho Data Integration (aka Kettle), but I was never a big fan. I couldn't find something that would make integrations easy, and intuitive. I've tried a lot of tools in the past, from the "must-see" players like Zapier and Make (integromat) going through some of the open source tools like n8n, Apache Airflow, Apache HOP, and a bunch of others which you can find at the end of this article.
Node-RED defines itself as a programming tool for wiring together hardware devices, APIs, and online services in new and interesting ways. It provides a browser-based editor that makes it easy to wire together flows using the wide range of nodes in the palette that can be deployed to its runtime in a single click.
Internally, we seem to be pushing Jitsu these days, but mostly because we have our sites on the "mass ingestion of events" not because it fits this category. Mage has caught my attention but it seems overkill for a lot of the "simple" moving information around, that we normally need (much like Jitsu does).
This past week I had a task to read from an Apache Kafka queue and send an HTTP request to a Cumulocity endpoint, so I started searching for a tool where I could create a Cumulocity connector that would give me a nice environment to work in and some additional functionality.
After a while, I had reduced the list you can find below down to:
I already had made my decision to work with prefect. It produced the best sensation over the rest, but I had to make sure so I started with the mindset of discarding the others. StackStorm and Luigi were easy, the first because it is far from the market segment of my project and the second due to the complexity. Then I set to look at Node-RED and justify why I wouldn't use it instead of prefect, and, boom! it hit me in the face. Wow!
Incredible, how had I not seen this tool before? Easy to install, the debugging and simplicity mixed with the power is mind-blowing. I had an Apache Kafka consumer sending emails when I produced messages in a few clicks. What?!?
Thousands of nodes existed so I searched for Cumulocity > it is already there !!
I have to say that if, for some reason, I couldn't use Node-RED for my project I would use prefect, but I am going to use Node-RED for this one and think I will be using it a lot in the future.
Once I had watched the video series, browsed around the forum, and read most of the documentation. I implemented the lowercase development example node and did what I usually do when I find an open-source product I like:
Translate it It is astonishing how much you can learn about an application by translating it. It is like going through every part of the application with notes. Much like reading the manual. The team at Node-RED has a nice policy where a translator needs a validator to review the translation before it can get accepted, so if any reader of this post can give me a hand there, it would be much appreciated.
Join the community: forum and Slack.
Implement an integration with coreBOS, obviously :-)
coreBOS Integration
The experience was, again, satisfying. The documentation, examples, community support, and overall development environment were positive. I had an integration with 11 nodes in less than a day, mostly due to my lack of comfort with node.js.
I recorded a couple of videos to present the integration.
Contenders
