object tracking with image sensor + FPGA?

Go To Last Post
8 posts / 0 new
Author
Message
#1
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Hi all, I'd like to make a robotic vision subsystem that behaves similar to this:
http://www.youtube.com/watch?v=tbHWvPWhVh8

I can envision an image sensor filling a SRAM or SDRAM under control of an AVR and then doing basic "what direction is the yellow ball? computations in the FPGA. What I can't seem to find is a starting point for a design.

Requirements: Run this at low resolution (e.g. 320x240) and 10 fps. Divide the viewable area into, say 25 rectangles and have the FPGA report the position of the colored object in 5 bits.

Can anyone help me with a starting design, sourcing of a suitable image sensor, etc? Thanks!

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Why do you need a fpga? The AVR itself could most likely solve your problem given your specs. Something like the CMUCam perhaps?

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I was thinking of an FPGA for two main reasons:

1. Even a 320x240 image at RGB 5:5:5 ends up being a lot of bytes (144,000) and I don't know of a mega-class 8-bit mcu that can address that much RAM.

2. The image processing needed (probably media filter followed by a threshold filter) at even 10 fps sounds like too hard a task for an 8-bit AVR. But an igloo-class FPGA could likely crunch through that in a few microseconds.

I'm willing to be proven wrong, open to any ideas.

Because of these reasons, I was thinking the proper role for an AVR in this case might be to use a timer to trigger the image sensor, then trigger the FPGA, then receive the result from the FPGA and report the result back to a larger system using SPI. "Desired colored object appears in region x of the image." Which might result in a command to "turn left and look down." That sort of thing.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

May I repeat myself:

http://www.cs.cmu.edu/~cmucam/

is this close to what you want??

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Have you checked out the Beagleboard?
Check out this link: http://mechomaniac.com/node/72

I've written a very robust object tracking program based around functions provided by OpenCV - however, it has been designed to run on a PC, not an embedded system.

I'd definitely investigate the OpenCV+Beagleboard+PS3Eye combination if possible. But I'm sure there are cheaper/easier ways to do what you'd like to do (I'm just biased towards OpenCV!) :D

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

@kartman, Sorry to not have responded to your suggestion earlier. I have looked at that device and yes, it seems close to what I'm looking for, but I wonder...
It seems strange that every listed US vendor seems to have a different version of that device on their websites, and the "what's new listing" is more than three years old.
I'll have to really look it over (some version of it anyway) before making a decision. I'm totally new to the topic of machine vision so I'm doing quite a bit of surfing. Thanks for the recommendation though, it is as I asked, a place to start. :)

@dmginc, I did evaluate the beagle for another (work-related) project some months back, but not in this context. By my remembrance, I disqualified it because its power budget was too high for that project. Can you remark on its power needs in your robot? Your application looks almost exactly like what I'd need at least to start with. How many frames per second can you process while tracking?

Thanks everyone, I'm still listening....

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

jhelmick wrote:
@kartman, Sorry to not have responded to your suggestion earlier. I have looked at that device and yes, it seems close to what I'm looking for, but I wonder...
It seems strange that every listed US vendor seems to have a different version of that device on their websites, and the "what's new listing" is more than three years old.
I'll have to really look it over (some version of it anyway) before making a decision. I'm totally new to the topic of machine vision so I'm doing quite a bit of surfing. Thanks for the recommendation though, it is as I asked, a place to start. :)

@dmginc, I did evaluate the beagle for another (work-related) project some months back, but not in this context. By my remembrance, I disqualified it because its power budget was too high for that project. Can you remark on its power needs in your robot? Your application looks almost exactly like what I'd need at least to start with. How many frames per second can you process while tracking?

Thanks everyone, I'm still listening....

That link I gave you wasn't actually one of my projects :D

In regard to the tracking algorithm I have designed however, I can track multiple fast-moving objects @ 30FPS+ (without multi-threading - which I still need to implement). As it only runs on a PC, I don't think it'll be suitable for your application though.

If you do decide to go down the OpenCV road, I'm happy to give you advice where possible!

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0