is possible? heterogeneous common bus input, common bus output protocol

Go To Last Post
16 posts / 0 new
Author
Message
#1
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

TLDR:

  • I need to make a network of arduinos that will receive data, process it and send it to other output arduinos.
  • The destination and origin of the messages is given by wether they are plugged.
  • They must allow many inputs and outputs

 

Explanation:

 

I have been studying and struggling on how to make a kind of distributed processing environment using arduinos, and of course, the communication protocol is the hardest part. I tried to make my own protocol with a simpler scheme, but I learned that I would need a lot of low level knowledge on AVR and C programming that I don't have. For the sake of prototype I am thinking to use ready made libraries, but it is not so easy yet:

I describe below what I am trying to do in terms of communication. But an image will make it way easier:

[see attached image]

I mainly need each arduino to have n possible data inputs, and n possible data outputs. The best would be that n is 'virtually unlimited', but it will be ok for now if n is just more than 1, in which case I could do a module that forks the stream, effectively making a virtually unlimited amount of outputs (at cost of speed), but that is not well desired.

*The data outputs can all be the same, as the output value will always be the same to all the "children". However, some "children" may be busy, meaning that perhaps it is easier to keep outputs independent.

The devices are dinamically plugged and unplugged, but they can be already powered then, avoiding hot plugging problems (thanks, grumpy_mike http://forum.arduino.cc/index.ph... )

I also need the "network" to be "heterogeneous", meaning that hopefully it doesn't need one single master, specially because there are many isolated communication nodes. If I am a node, I know what I am sending to my children node, but I have no idea of what that "children" node is sending to my "grandchildren" node.

If interested, an example application is the one in this video (https://www.youtube.com/watch?v=...), only that in hardware, with all the changes that it implies to the idea.

So, a thought solution was that each device (arduino) would use the I2C protocol twice. Each device is both a slave and a master: it is a master to all the devices down, and a slave to all the devices up. Would this work? or will I be wasting my time, as I have been quite some days already?
if so, what is the best solution?
Am I dreaming of making a rocket to mars?

 

I know that mega readily does that, but I think it is on the expensive side, and also it has too much processing power that will be wasted, and that makes me sad. Whilst I will try to avoid it, if it is too much of a hassle, I can go for it.

 

[note: I posted the same question in the arduino forum http://forum.arduino.cc/index.ph...

Last Edited: Fri. May 26, 2017 - 10:47 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 1

You could try i2c, but I'd suggest  you'll tear your hair out getting it to work reliably. The technique that comes to mind and would be extensible and reliable is to use ethernet. You can get wiznet modules fairly cheap and enc28j60 modules probably cheaper. Ethernet manages the issues of multiple devices trying to talk at the same time and as well, PC and other devices can communicate as well.

Another solution might be CAN protocol - it too manages the multimaster situation in hardware. RS485 is do-able but it requires a fair amount of work as you have to devise a mechanism to handle multi-master and collisions.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Kartman wrote:
The technique that comes to mind and would be extensible and reliable is to use ethernet.

Indeed.

 

And, with IP, it then it becomes trivial if you want to go wireless and use WiFi, Cellular, or whatever ... 

Top Tips:

  1. How to properly post source code - see: https://www.avrfreaks.net/comment... - also how to properly include images/pictures
  2. "Garbage" characters on a serial terminal are (almost?) invariably due to wrong baud rate - see: https://learn.sparkfun.com/tutorials/serial-communication
  3. Wrong baud rate is usually due to not running at the speed you thought; check by blinking a LED to see if you get the speed you expected
  4. Difference between a crystal, and a crystal oscillatorhttps://www.avrfreaks.net/comment...
  5. When your question is resolved, mark the solution: https://www.avrfreaks.net/comment...
  6. Beginner's "Getting Started" tips: https://www.avrfreaks.net/comment...
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Both ideas were out from my current field of view; which is nice. Many thanks! I feel a bit of respect (read fear) for the net protocol; for the case of the enc28j60, (*1)would it be possible to connect many devices to the line, as you would do with I2C, or would you need to have one enc28j60 per terminal? It would be quite nice being able to use networking to communicate among MCUs.

 

The CAN protocol looks really sweet. I am seeing about the MCP2515 chip that is available and well documented. However,(*2) how would you go about to segregate the communications? This protocol is build to make all the devices connected together, but in my case, I need each node to only know about his surrounding network; not the whole network. I guess this question goes as well for the enc28j60 protocol.

The two ways of doing this that I can think, are

  • that each module would have two MCP2515 chips, one for the inputs and one for the outputs, thus isolating both of the networks. It will be a big over-spec. But who knows, maybe in the future I will need to abuse of the bandwidth of a single node's output/ input. (*3) I don't know however, if programming the Arduino to handle two MCP's will be to hairy.
  • that all share the same bus. The contras are that I will need to figure out how one module will know that is the ID of the module that is supposedly connected to. There will also be a lot of bandwidth waste, as all modules will receive signals that they actually don't need. Probably in the case of having to use a common bus for all modules, will mean that I have to change my original design idea.

What are your better educated opinions, Kartman and awneil? (or well, actually anyone who knows)

I put these (*n) numbers to make it easier to locate particular questions. I hope this is a topic you enjoy discussing about :P

Many thanks!

Last Edited: Sat. May 27, 2017 - 09:02 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

With the enc28j60 or wiznet module the networking is exactly the same as you'd network your PC - you need a hub or switch to connect each of your nodes to the network. Whilst it might be a bit more intimidating, you'd be using the most used protocol on earth, so there's no shortage of information. You could also prototype it on PCs then transfer the learning onto the Arduino boards. Basically this is what all the hype about IoT is - connecting little micros to talk to big computers via the interwebs. You want to connect little micros to other little micros, so the basics are all the same. Being able to listen/ send messages from your PC will help your debugging.
Then we have CAN. Thiswas developed by Bosch for use in automobiles. It has also extended to industrial controls. With CAN you send small messages that have a priority and an ID - the priority is used to resolve who wins when two or more nodes transmit at the same time and the ID is used by the other nodes to determine if they want to listen to your message or not. In your case each of your nodes might send a message with their input status but with a unique ID so we know who sent it at a regular interval. Your nodes could also decide what messages to listen to and copy to their outputs.
So, in answer to your questions - with CAN you would probably only need one interface - i think two might be overkill and you'd need to think about routing between busses. As to figuring out who needs to listen to what - that is the challenge.

Last Edited: Sat. May 27, 2017 - 12:16 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Ok, so that rules the networking possibility out, because I plan to have many of these devices connected forming many small networks. That would mean that I would need to have many switches. It would be great that there was a way of making a low level switch for basic serial communications. Actually my first attempt, was to use a mux as a switch for serial, but it got very complicated quickly. My plan is to make cheap, small elements, so that's why I didn't just chose raspberries or arduino megas.

Doesn't it exist at all, some sort of Serial switch that can be on the board?

 

The CAN chips seem to be ok price and size, and now I am thinking on ways that I can try that in a protoboard. Can does use quite large messages anyhow. I am pondering wether the tradeoff of message lengths that CAN implies is superior to the tradeoff of using arduino megas.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I'm a little confused as to the exact aim you have. I'm assuming that with a mega, you have four uarts. Thus each of your links is one to one. Thus a given board could have one unique link to one or more (up to four boards). So for a given message to get from A to B might require traversing through a number of boards.this sounds a bit like the telephone system in that you need an addressing and routing mechanism and the routing could be via one or many unique routes.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Kartman wrote:
you need an addressing and routing mechanism

Which, surely, is all part of IP ?

 

Top Tips:

  1. How to properly post source code - see: https://www.avrfreaks.net/comment... - also how to properly include images/pictures
  2. "Garbage" characters on a serial terminal are (almost?) invariably due to wrong baud rate - see: https://learn.sparkfun.com/tutorials/serial-communication
  3. Wrong baud rate is usually due to not running at the speed you thought; check by blinking a LED to see if you get the speed you expected
  4. Difference between a crystal, and a crystal oscillatorhttps://www.avrfreaks.net/comment...
  5. When your question is resolved, mark the solution: https://www.avrfreaks.net/comment...
  6. Beginner's "Getting Started" tips: https://www.avrfreaks.net/comment...
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Routing with only 2k ram on a mega328 is going to be rudimentry at best.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

yes! that is the idea... I am realizing how bad I am at explaining and contextualizing. I apologize.

Let's imagine I want to do this:

 

example application

 

Where you started pressing the a button on the module that says bang, which sent a zero to the module that was connected downstream. This byte started propagating down, and changing depending on the module that it went through.

 

Well, making a modular calculator is a bit of an absurd application. My real intention is to make this, but with midi alike messages, and each module will have whatever user interface, and make changes to the midi note accordingly. It is not the type of communication system that you want to attain speed, but I expect that in the end there will be an element that is waiting for a midi clock signal to send the message, so the whole system is actually generating the data that will come out on the next clock and so on. But that is not the part I am most stuck with.

The idea is that plugging and unplugging these elements is something similar to patching modules in an eurorack modular rack; only that in this case it is digital data.

 

 

Last Edited: Sat. May 27, 2017 - 02:16 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I'm still having trouble getting the general idea of what you want to achieve.

with a modular synthesiser you patch things like audio and control voltages. Each module usually has an input(s) and an output and will modify the inputs in a given way to give an output.

If we wind back the clock, there were things called analog computers which were a bit like a modular synth in that you had various blocks that did 'things' like multiply, add and the like.  The inputs and outputs were voltages and you patched the various modules together to perform the equation you wanted and then you provided the inputs. A voltmeter was used to read the result. In your case  are you proposing something similar in that the voltages are replaced with a digital value? Now I think I'm starting to see what you're proposing. As to why you might want to have separate modules to do this, I don't know. I would think it would be simpler to use the concept of the different modules but implement them as virtual modules on the one device. Or at least prototype it that way.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I did one single device that has a virtual modular environment inside, on a raspberry. It has an interface with buttons, and you can page between the modules, and connect them. It is totally an interesting thing that I am still expanding.

 

 

I also did a digital prototype for the idea that I am now trying to do physically (here) In the near future I want to implement this virtual prototype as a sort of back-end to design how the single device prototype operates. But I want to explore in parallel with physical elements, mainly because it is easier for people to understand the concept of "this objects that you plug together" than "a device that has a virtual modular environment that you can design using a client over http and using a single interface". Also these things may be used for music performance, so the physical objects allows a better interaction "bandwidth" than a virtual environment.

My final goal is to make a modules that you could actually buy/diy and integrate among; and keep creating new modules in the future.

 

I tend to think that there should be a way to multiplex a UART port if you can program the communication protocol on all of the ends. Today, searching a lot I realized that there is no ready made communication protocol for such a kind of network. Probably because it will never be the best way to communicate all the devices together.

 

I thought of a solution that I will be testing tomorrow, that consists on using a multiplexor, and having the nodes downstream to ask permission to send a message by setting a flag. I wrote the firmware just to think about it in more detail. (here) Tomorrow I should probably go step by step from simplest to more complex, until I get it working... hopefully. Otherwise I may think on just using the arduino Mega.

 

I actually was expecting that there would be a simple answer, like an cheap IC that would record any serial input and keep it (this, but cheap) or a protocol that can be applied easily with a library, as it is the case with I2C. I will keep posted on how it goes! And I can't thank enough for all the help that you guys have been giving me on this. :)

 

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

When thinking about CAN, it helps to remember that you do not send message from one place to another but rather that you send a message to the whole world and that individual nodes decide if that message is for them or not.

#1 This forum helps those that help themselves

#2 All grounds are not created equal

#3 How have you proved that your chip is running at xxMHz?

#4 "If you think you need floating point to solve the problem then you don't understand the problem. If you really do need floating point then you have a problem you do not understand." - Heater's ex-boss

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

The problem with implementing the comms literally is that if you want clock messages, they would have to propagate through the maze. Not impossible, but i think it might create synchronisation issues. So, if we use a comms bus technique, like CAN, we have an easy method of sending messages to one or many nodes at once. Conceptually though, we need a method of routing the messages to create a virtual patch on the bus. One method i conceived was to use i2c To communicate the module's ID. You can easily bit- bash i2c masters, so on a mega328 for could implement an extra i2c master. So the idea is that the physical patching is done with patch cords carrying i2c. So a bit bashed i2c master OUT on one module patches to a hardware i2c IN on another module. The OUT asks an IN for the ID. This only needs to be done once when they are patched. Once the OUT knows to destination ID, he can send messages with that ID and the IN module is listens to messages with his ID. Basically pub-sub via CAN.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

True. But what I liked about CAN is that you can buy an inexpensive, dedicated IC. I could potentially provide each module with two of these (one for inputs, other for outputs).

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

You understand the idea very well. I was a bit reticent about using a common bus for all the modules. At the same time I was discussing this with some peers, and they suggested me the exact same idea. I feel that it takes out a bit of the magic, but in exchange, you can monitor the state of all the communications by monitoring only one bus, which is nice. I couldn't try ideas today, but during this week I will be trying all the approaches that I can: common I2C bus, arduino mega, multiplexed serial and the weird idea of splitting each module into two networks.

There is no problem with the clock signal, because only some particular nodes need it; and can be provided by a dedicated cable that the user chooses to patch. It also would allow to have many independent clocks, if that would make sense in any circumnstance.

So, I will not be posting much until I have done tests, and then  I will come back, hopefully with a better solution to share. Many thanks for all the answers, and for the effort that it took to understand the problem; specially Kartman and awneil.

Hopefully I will also be able to show more about the virtual implementation of this modular thing, if you are interested.