AVR Freaks

I can remember at the Fibre Optic company two Engineers had been working on an audio noise issue for over a week.  They could not figure out where the noise they were hearing in the signal was coming from, and why it varied with the amount of cards in the subrack,  although they suspected it was in the power supply of the unit itself.  They were getting very frustrated and the customer was losing their patience.  I generally stayed out of the way, but I was asked to have a look see/hear see.   I put on the headphones and knew exactly what the problem was.  The engineers were on the right track, but on the wrong train.  It was in the power supply like they thought, but not the power supply on the unit.  It was the hunk of junk switching power supply the VP of Engineering designed and insisted we used.  The damned things switching frequency shifted between 1Khz and 3Khz!  I simply smiled and pulled the SMTP from the subrack and threw it in the trash in front of them - And the VP of engineering -, took one of the heavy duty supplies we used for the larger systems and powered things up with that...Tada!  pesky noise GONE!

They picked up my Happy Hour tab that night.

Side note about those in-house designed supplies....  THey were so unstable that they often exploded during final testing/output voltage setting.  Me being the youngest in the company was given suicide task of performing the final tests as I had the youngest heart....bastards.

Jim

OK, I have several, but will limit it to one.

The Saudi Met service incorporated a section that was responsible for creating a geographical information system/responsibility. They had hired a few "experts" to design and institute it. One thing they did was to order some fancy piece of electronics from the US. It was impressive if only in size. A single pcb literally 2 feet by 3 feet crammed with TTL devices. This was 1983 or 1984.  On powering up it didn't do whatever it was supposed to do and "they" had poured over it for hours trying to determine the fault. Tim, the Australian computer guru was invited to give an opinion. He then came down to my section and I was "sub-contracted" to share the group wide ignorance.

The first thing I noticed was the shear size of the beast and the dozens of same coloured mod wires that hung above the pcb like a swarm of blow flies over a dead carcass. Not a single one was anchored to the pcb except by solder at each end. That was my opening thought. So I then proceeded to visually trace every mod wire from one end to its other. Eventually I found the unattached end floating in the air. Then it was a close examination of all the solder points in the general area beneath the flying end. Eventually the telltale cavity in a solder pad was found.

I was not willing to assume that I was correct so documented the start and end locations and these were telexed (remember them?) to the US manufacturer who subsequently confirmed that solution.

Someone else then had the pleasure of gooping each and every mod wire onto the pcb to prevent a repeat.

I am still waiting for my happy hour bill  to be met.

No tour?

One more from me, but nothing to do with me.

On Christmas Day 1974, Cyclone Tracy swept over the Australian city of Darwin killing 71 people and flattening the city and the Met Bureau's facilities for monitoring the weather further. Part of the review concluded that we needed a rapid response, portable and self contained means of providing weather data following similar disasters. Relief aircraft needed surface wind and pressure data for safe arrivals and departures.

So a long body caravan was chosen that was to be loaded into a C130 Air Force transport. All of the relevant sections of the Met Bureau put in their space, weight, power and exposure requirements for their equipment to the only mechanical engineer on staff. "Robo" worked assiduously and after about 6 months a fully equipped prototype had been created and fitted out for its first trial. Sadly it did not go well. The loading ramp was not level and when the caravan was being pulled into the C130 its leading edge would not clear the top of the opening door. The height of the caravan was greater on a slope than on level ground. Ooops. "Robo" retired shortly afterwards.

I'm usually only involved in the software side of things but one of the major hardware screw ups I know about concerns the best part of 2 million satellite TV recorders that were installed in UK user's homes. Now customers just use these things to pick some programs from a program guide and say "record that, that and that when they are broadcast". This builds a library of recordings then at their leisure you can go back to the list of recordings and playback the things at a time that suits you. Suppose you record a 6 part crime drama but just as it is reaching the denouement in episode 6 a messsage pops up to say "there is a technical fault - press Select to skip corrupted section" and you therefore don't get to see who actually did it. How frustrated you would be? In a more mundane way perhaps you just record a weekly soap and find that every third episode has corruptions and recording drops outs. When something like 10-20% of the 2m units fail (200,000 .. 400,000) then that makes for rather a large number of very unhappy customers who are typically paying anything from £20 to £60 per month for the privilege of being able to view and record several hundred channels.

It turns out that in the sat TV recorders there are a couple of Network Interface Modules (NIMs) that are the things that take the 1.4GHz (is it) signal off the 11GHz carrier and then do the QPSK demod/decode to retrieve the 27MB bit stream that contains the 10+ channels on one transponder link. Beyond that, in the box, it's all just about digital recording but these modules interface to the Low Noise Block. To take the digital data off the carrier it is absolutely vital that the Vcc to these module is rock solid with no ripple whatsoever.

Now it turned out that when you are making 2 million or more units of something that if you can save just $0.05 on the cost of a couple of electrolytics or something then it's worthwhile doing ($0.05 * 2,000,000 = $100,000). Sadly the guy who designed the PSU was a bit "gung ho" and eager to please everyone so he specified some particularly cheap electrolytics in the PSU design.

Oh and another thing you should know about satTV recorders is that they generally all center around a 3.5" hard drive where all the program data gets stored. These mechanical things suffer from friction and it's not unusual for them to run at 65degC or more. Also, because loads of people have their satTv recorder in their bedroom it's imperative they don't make much noise - so while there may be a small fan for a minimal amount of forced air cooling you cannot have large/noisy cooling systems.

It transpires that if you position cheap PSU electrolytics close to the fierce heat source that is a hard drive in such a poorly cooled box that they "cook". In time the electrolytic liquid dries out and they lose their capacitance and ability to smooth the ripple on the 5V line that goes to power the NIMs.

When the NIMs start to receive a "dirty" power line they get quite sporadic in their ability to decode the digital from the sat signal.

Another aspect of many (and this) sat TV recorders is that there are two NIMs. They tend to alternate in operation - so one may be used to present "live TV" while the other is involved in recording scheduled shows. So if they are just at the "borderline" of work/fail it can well look like the "viewing" NIM is presenting a "clean" picture while the recording one is putting bursts of MPEG2 dropout/noise into the thing being recorded. So when the engineer visits it can appear OK but "healthy" but with the actual problem "hidden"/sporadic.

The satellite broadcaster we sold all the boxes to - who was the one having to deal with 300,000+ irate customers demanding a refund of their monthly subscription - was not amused! (to say the least!!)