Craziest Encounter

Go To Last Post
23 posts / 0 new
Author
Message
#1
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Wondering: what was the craziest snafu you've been presented with?

 

I was called in to debug a signal mux/data acquisition monitor fixture that had several PC-sized units interconnected with a boatload of cables and connectors between them.  It wasn't completely dead, but nothing it did made much sense.

When I started digging into the issue, I found that their schematics had a number of wiring mismatches (between the schematics themselves).   Pressure sensor A in one box, accidentally & supposedly became Temperature sensor A in another box.  Apparently their tech had raised the issue during construction, but was told it would be taken care of later & he should concern himself with just building what he was given.   However, that was only the beginning of the madness.  Some circuit boards layouts had some of their connectors mirrored.  In some cases this was "ok" since the other cable end had accidentally been mirrored as well.  But tapping into the cable itself for a reading would give the wrong information.  Others had the physical circular connectors rotated off by one pin count.  Making my way through this maze (thanks a lot for using all the same color wires), was quite an adventure.  Luckily no damage was done by all of this.  It finally came together, after most of the original wiring was completely trashed.

 

When in the dark remember-the future looks brighter than ever.   I look forward to being able to predict the future!

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I was working for a company that produced optical character readers, from large high speed check sorters to small handheld price tag readers for retail stores.   One project I worked on was reducing the analog front end, all built with discrete components and taking several square inches of pcb space down to a single 24 pin IC.  The prototyping went very well, when the first run of new chips came in however and was placed onto the new pcb that came from manufacturing, the unit failed to work.  All morning the engineers came in, picked up a scope probe and examined all the signals on the pins of the new chip.  Things just did not look right, so with the team of engineers gathered in a huddle discussing what could have gone wrong with the new chip. I walked over to examine the new board, within a minute, I had identified the problem.  Manufacturing had mixed up one of the parts, a 10pf cap (marking 100) had been placed where a 100pf (marking 101) should have been, resulting in improper ADC operation.  All had looked at the caps and seen 100 marked on them and assumed they were the correct part.  Needless to say, my lunch was free that day!

 

Click Link: Get Free Stock: Retire early! PM for strategy

share.robinhood.com/jamesc3274
get $5 free gold/silver https://www.onegold.com/join/713...

 

 

 

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I can remember at the Fibre Optic company two Engineers had been working on an audio noise issue for over a week.  They could not figure out where the noise they were hearing in the signal was coming from, and why it varied with the amount of cards in the subrack,  although they suspected it was in the power supply of the unit itself.  They were getting very frustrated and the customer was losing their patience.  I generally stayed out of the way, but I was asked to have a look see/hear see.   I put on the headphones and knew exactly what the problem was.  The engineers were on the right track, but on the wrong train.  It was in the power supply like they thought, but not the power supply on the unit.  It was the hunk of junk switching power supply the VP of Engineering designed and insisted we used.  The damned things switching frequency shifted between 1Khz and 3Khz!  I simply smiled and pulled the SMTP from the subrack and threw it in the trash in front of them - And the VP of engineering -, took one of the heavy duty supplies we used for the larger systems and powered things up with that...Tada!  pesky noise GONE!

 

They picked up my Happy Hour tab that night.

 

Side note about those in-house designed supplies....  THey were so unstable that they often exploded during final testing/output voltage setting.  Me being the youngest in the company was given suicide task of performing the final tests as I had the youngest heart....bastards.

 

Jim

I would rather attempt something great and fail, than attempt nothing and succeed - Fortune Cookie

 

"The critical shortage here is not stuff, but time." - Johan Ekdahl

 

"Step N is required before you can do step N+1!" - ka7ehk

 

"If you want a career with a known path - become an undertaker. Dead people don't sue!" - Kartman

"Why is there a "Highway to Hell" and only a "Stairway to Heaven"? A prediction of the expected traffic load?"  - Lee "theusch"

 

Speak sweetly. It makes your words easier to digest when at a later date you have to eat them ;-)  - Source Unknown

Please Read: Code-of-Conduct

Atmel Studio6.2/AS7, DipTrace, Quartus, MPLAB, RSLogix user

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

jgmdesign wrote:
Side note about those in-house designed supplies....  THey were so unstable that they often exploded during final testing/output voltage setting.  Me being the youngest in the company was given suicide task of performing the final tests as I had the youngest heart....bastards.

+1  been there, done that too!

Click Link: Get Free Stock: Retire early! PM for strategy

share.robinhood.com/jamesc3274
get $5 free gold/silver https://www.onegold.com/join/713...

 

 

 

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

OK, I have several, but will limit it to one.

 

The Saudi Met service incorporated a section that was responsible for creating a geographical information system/responsibility. They had hired a few "experts" to design and institute it. One thing they did was to order some fancy piece of electronics from the US. It was impressive if only in size. A single pcb literally 2 feet by 3 feet crammed with TTL devices. This was 1983 or 1984.  On powering up it didn't do whatever it was supposed to do and "they" had poured over it for hours trying to determine the fault. Tim, the Australian computer guru was invited to give an opinion. He then came down to my section and I was "sub-contracted" to share the group wide ignorance.

 

The first thing I noticed was the shear size of the beast and the dozens of same coloured mod wires that hung above the pcb like a swarm of blow flies over a dead carcass. Not a single one was anchored to the pcb except by solder at each end. That was my opening thought. So I then proceeded to visually trace every mod wire from one end to its other. Eventually I found the unattached end floating in the air. Then it was a close examination of all the solder points in the general area beneath the flying end. Eventually the telltale cavity in a solder pad was found.

 

I was not willing to assume that I was correct so documented the start and end locations and these were telexed (remember them?) to the US manufacturer who subsequently confirmed that solution.

 

Someone else then had the pleasure of gooping each and every mod wire onto the pcb to prevent a repeat.

 

I am still waiting for my happy hour bill  to be met. devil

Ross McKenzie ValuSoft Melbourne Australia

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Comparing the previous stories, much to my surprise, there has never been a technical SNAFU in my life. Ever.

 

And very few non-technical ones. In fact, the only one I can think of, in any detail, I lay responsibility on my dear youngest brother. He and I had been hiking small sections of the Pacific Crest Trail in southern Washington. We had finished just over 100 miles in several years of work and were about to start the penultimate section of the southern Washington Cascades - an incredible area called Goat Rocks. 

 

We drove to the trail-head without much problem. At least, we THOUGHT is was the trailhead. My brother insisted that his GPS showed the spot where we were as the trailhead. And, there was, indeed, a trail there. This was supposed to be a short side-trail that intercepted the Pacific Crest Trail in about a mile. We set out, and the trail seemed to go in the right direction, but gradually veered away from where we expected it to go. We stopped a bit and my brother pulled out his GPS. "Oh, sh*t! This isn't the trail we want! The PCT is over that way, about 1/4 mile." It was fortunately pretty open, not much underbrush and such, and it became clear that we were not the only ones to walk this way. In a few minutes, the PCT was found, and we were on our way.

 

So, it ended up being "not really SNAFU" but it could have been a lot more unpleasant than it was. Pretty unexciting. Goat Rocks was exciting. It lived up to expectations.

 

Jim

Jim Wagner Oregon Research Electronics, Consulting Div. Tangent, OR, USA http://www.orelectronics.net

Last Edited: Tue. Aug 6, 2019 - 03:40 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

ka7ehk wrote:
Goat Rocks was exciting. It lived up to expectations.

 

No tour?

I would rather attempt something great and fail, than attempt nothing and succeed - Fortune Cookie

 

"The critical shortage here is not stuff, but time." - Johan Ekdahl

 

"Step N is required before you can do step N+1!" - ka7ehk

 

"If you want a career with a known path - become an undertaker. Dead people don't sue!" - Kartman

"Why is there a "Highway to Hell" and only a "Stairway to Heaven"? A prediction of the expected traffic load?"  - Lee "theusch"

 

Speak sweetly. It makes your words easier to digest when at a later date you have to eat them ;-)  - Source Unknown

Please Read: Code-of-Conduct

Atmel Studio6.2/AS7, DipTrace, Quartus, MPLAB, RSLogix user

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

No tour! They liked to keep to their own herds.

 

Jim

Jim Wagner Oregon Research Electronics, Consulting Div. Tangent, OR, USA http://www.orelectronics.net

Last Edited: Tue. Aug 6, 2019 - 05:30 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I worked for a medical company that found one unit of our product had an unusual buzz.  I tried everything that I could think of, but could not find the source.  Our senior technician took the unit apart board by board and we closely examined everything, but no luck.

 

We were finally left with nothing but the mains cable, a standard NEMA 5-15P_to_IEC C13 (PC to Wall-outlet USA).  The senior tech did a continuity check and found that the Hot and Neutral were correct, but there was no connection between the third ground leads.   He cut the cable only to find that there was no wire at all between the ground pins: two-wire cable attached to three-wire connectors.

 

Since we supplied the cable with the unit, we had to contact every hospital that had this model and have them test their cable.  Then we had to file an official product recall with USA Federal Food and Drug Administration, a complete nightmare.

 

Just another day in the quality control department of the "One Hung Low" electronics factory, Shenzhen, China.   I used to think that the American technicians were too hard in their criticisms of Chinese manufacturing quality.  Now I double check everything that arrives from China.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

the mains cable, a standard NEMA 5-15P_to_IEC C13

These days a lot of the 120 V Mains power cords for medical devices have a clear plastic molded connector with the pins.

That way one can see the three wires, including the nice green one.

The Green, Ground, wire is always longer than the Hot and neutral wires, with some "extra" being crammed into the connector.

That way, if the cord ever weakens and wires are pulled out from the connector, the Ground wire will always be the last one to fail.

 

Perhaps it was episodes such as the one you mentioned that lead to such practices?

 

JC 

 

Edit:

 

Web photo:  C2G 16 AWG Hospital Grade cord, (NEMA 5-1P...)

 

Image result for medical power cords

Image result for medical power cords

Last Edited: Tue. Aug 6, 2019 - 09:09 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

DocJC wrote:

The Green, Ground, wire is always longer than the Hot and neutral wires, with some "extra" being crammed into the connector.

That way, if the cord ever weakens and wires are pulled out from the connector, the Ground wire will always be the last one to fail.

Most plugs does have the terminal of ground wire "closer" than hot and neutral, less chances for human error that way, since then all wires can be same length, but the design quarantees ground is the last one connected(well atleast here in EU)

Last Edited: Tue. Aug 6, 2019 - 09:37 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I was asked to look at a machine that was acting funny. It would work only if a piece of cardboard from the back of a pad of paper was laid across the top circuit board in the rack. If you removed it, or turned it 90 degrees, the machine would stop.

 

Turned out to be an unsoldered pin on a 74LS type IC.

 

(About 1983)

The largest known prime number: 282589933-1

It's easy to stop breaking the 10th commandment! Break the 8th instead. 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

 He cut the cable only to find that there was no wire at all between the ground pins: two-wire cable attached to three-wire connectors.

 

I was gong nutty trying to download some data from my phone to the PC...Only when I switched USB cables did it work...Found out, to save money, the Ebay cable I used did not include the data lines!!!....arggg...Now I know.  I had seen similar for misc wires in DB-9 cables, but didn't expect it for USB.

 

 

When in the dark remember-the future looks brighter than ever.   I look forward to being able to predict the future!

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Take a look at the eBay advert. It might say USB charging cable and not data cable.

Jim

I would rather attempt something great and fail, than attempt nothing and succeed - Fortune Cookie

 

"The critical shortage here is not stuff, but time." - Johan Ekdahl

 

"Step N is required before you can do step N+1!" - ka7ehk

 

"If you want a career with a known path - become an undertaker. Dead people don't sue!" - Kartman

"Why is there a "Highway to Hell" and only a "Stairway to Heaven"? A prediction of the expected traffic load?"  - Lee "theusch"

 

Speak sweetly. It makes your words easier to digest when at a later date you have to eat them ;-)  - Source Unknown

Please Read: Code-of-Conduct

Atmel Studio6.2/AS7, DipTrace, Quartus, MPLAB, RSLogix user

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I've had a few in my time. Some technical, some environmental but most caused by lack of attention.

 

I was involved in a large lighting installation with over 2000 controlled channels. The electrical apprentices were left to their own devices whilst the foreman went to the pub. Testing involved turning each channel on/off and seeing that it agreed with the drawings. >30% didn't quite control what they should've. One channel had been wired to the ladies toilets. A lady who been using said conveniences was not happy the lights were turned off. That saved us trying to find where it went! It took a couple of weeks to work through all the problems. Thankfully I didn't have to crawl through roof cavities etc to follow wires.

 

 

 

 

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

had a couple.

When working in a small company on tight budget, there was a product that I was not involved with. The mechanical engineer had made a very nice housing with integrated cooling plate and a number of connections exiting from the PCB to each side. It looked really cool.

The owner of the company thought too and decided to invite a number of people to let the mechanical engineer show what he had made. everybody agreed that it looked great. Then he went on to show the exploded view and started to assemble things in his MCAD package.

I immediately saw a flaw in his PCB model in that it was just a square with nothing on it, while in reality it had connectors at 3 sides.

I let him continue as to see what else was going on. During the assembly process the PCB insert was a matter of milliseconds and was not addressed by the other engineers, even the ones that were working on the project. Then in not time some end caps were placed and presto the thing was assembled. The owner of the company had some other things to tell us, so at that point in time no discussion about this. ( the story was that we had to be very careful spending money as at that point a number of large payments (income for the company) were due so we were short on cash. Next was the end of meeting questions round. I said I had a couple when I got the word. First one was if the mechanical design was already ordered, answer was that all he had to do was press the send button on an e-mail and it would be done all else was done and ready. Then I asked to be shown the assembly process again as I thought something was wrong. The company owner (the one that had to send the e-mail) first made a couple of jokes about it and a couple of people joined in that I should have payed better attention. well my reply simply was that they should have paid better attention as they had been in one way or the other involved in the project. The mechanical engineer saw on my fase that I was serious, so he started the assembly part again. Now when we got to the part were the PCB had to be inserted/mounted and capped I tolled him to stop. Now I asked the engineers if this was the real PCB and how they envisioned how it was going to be produced. They said this was the PCB, so I said so the connectors will not go on until the PCB is mounted in the housing??? That gave a bit of a discussion, but the answer was no, so I tolled them that this would never work as the PCB could only be mounted without the connectors, or the cover plates could never be mounted. I did not get a thank you from the owner ( he did not like me so that was expected) but all the colleagues and specially the mechanical engineer give a big thanks. Saved 2 months developing time perhaps even more as it was going towards the holiday season and IIRC more than 30K in money wasted on a totally useless aluminum extrusion piece.

 

Second one was that I had a colleague that was very sensitive to high frequency audio (15KHz - 20KHz) and whenever I started a load test on a new product he would hear a very irritating noise. He did not mention it, but when we went to the customer with the first production boards to give them a demo. they noted that "it was almost as if it makes a strange noise" at that point the colleague triggered ( he was not involved directly, but in our HW synchronization meeting things like this were discussed). He tolled that when ever I started a load test he would also hear a strange noise.

It took us a while, but in the end it turned out that the capacitive boost regulator was the culprit. It excited the capacitors to such extend that they would start to emit audio, specially in low load conditions as then it reduced its working frequency from about 25KHz to just under 20KHz.

 

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

One more from me, but nothing to do with me.

 

On Christmas Day 1974, Cyclone Tracy swept over the Australian city of Darwin killing 71 people and flattening the city and the Met Bureau's facilities for monitoring the weather further. Part of the review concluded that we needed a rapid response, portable and self contained means of providing weather data following similar disasters. Relief aircraft needed surface wind and pressure data for safe arrivals and departures.

 

So a long body caravan was chosen that was to be loaded into a C130 Air Force transport. All of the relevant sections of the Met Bureau put in their space, weight, power and exposure requirements for their equipment to the only mechanical engineer on staff. "Robo" worked assiduously and after about 6 months a fully equipped prototype had been created and fitted out for its first trial. Sadly it did not go well. The loading ramp was not level and when the caravan was being pulled into the C130 its leading edge would not clear the top of the opening door. The height of the caravan was greater on a slope than on level ground. Ooops. "Robo" retired shortly afterwards.

Ross McKenzie ValuSoft Melbourne Australia

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I don't follow.    The C130 will have a specific height in its cargo bay.

 

If the caravan is at an angle due to the loading ramp,  the height is H cos x i.e. always less than H.

If you want to reduce the ramp angle,   you put the caravan on a low-loader trailer and adjust the trailer height.

 

I guess that these existed in 1974.    And would be available at most places in Australia.

You probably would not want to fart about with ramps, trailers, ... in a war zone.    But your natives should be friendly.

 

David.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I'm usually only involved in the software side of things but one of the major hardware screw ups I know about concerns the best part of 2 million satellite TV recorders that were installed in UK user's homes. Now customers just use these things to pick some programs from a program guide and say "record that, that and that when they are broadcast". This builds a library of recordings then at their leisure you can go back to the list of recordings and playback the things at a time that suits you. Suppose you record a 6 part crime drama but just as it is reaching the denouement in episode 6 a messsage pops up to say "there is a technical fault - press Select to skip corrupted section" and you therefore don't get to see who actually did it. How frustrated you would be? In a more mundane way perhaps you just record a weekly soap and find that every third episode has corruptions and recording drops outs. When something like 10-20% of the 2m units fail (200,000 .. 400,000) then that makes for rather a large number of very unhappy customers who are typically paying anything from £20 to £60 per month for the privilege of being able to view and record several hundred channels.

 

It turns out that in the sat TV recorders there are a couple of Network Interface Modules (NIMs) that are the things that take the 1.4GHz (is it) signal off the 11GHz carrier and then do the QPSK demod/decode to retrieve the 27MB bit stream that contains the 10+ channels on one transponder link. Beyond that, in the box, it's all just about digital recording but these modules interface to the Low Noise Block. To take the digital data off the carrier it is absolutely vital that the Vcc to these module is rock solid with no ripple whatsoever.

 

Now it turned out that when you are making 2 million or more units of something that if you can save just $0.05 on the cost of a couple of electrolytics or something then it's worthwhile doing ($0.05 * 2,000,000 = $100,000). Sadly the guy who designed the PSU was a bit "gung ho" and eager to please everyone so he specified some particularly cheap electrolytics in the PSU design.

 

Oh and another thing you should know about satTV recorders is that they generally all center around a 3.5" hard drive where all the program data gets stored. These mechanical things suffer from friction and it's not unusual for them to run at 65degC or more. Also, because loads of people have their satTv recorder in their bedroom it's imperative they don't make much noise - so while there may be a small fan for a minimal amount of forced air cooling you cannot have large/noisy cooling systems.

 

It transpires that if you position cheap PSU electrolytics close to the fierce heat source that is a hard drive in such a poorly cooled box that they "cook". In time the electrolytic liquid dries out and they lose their capacitance and ability to smooth the ripple on the 5V line that goes to power the NIMs.

 

When the NIMs start to receive a "dirty" power line they get quite sporadic in their ability to decode the digital from the sat signal.

 

Another aspect of many (and this) sat TV recorders is that there are two NIMs. They tend to alternate in operation - so one may be used to present "live TV" while the other is involved in recording scheduled shows. So if they are just at the "borderline" of work/fail it can well look like the "viewing" NIM is presenting a "clean" picture while the recording one is putting bursts of MPEG2 dropout/noise into the thing being recorded. So when the engineer visits it can appear OK but "healthy" but with the actual problem "hidden"/sporadic.

 

The satellite broadcaster we sold all the boxes to - who was the one having to deal with 300,000+ irate customers demanding a refund of their monthly subscription - was not amused! (to say the least!!)

Last Edited: Wed. Aug 7, 2019 - 09:32 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

david.prentice wrote:

I don't follow.    The C130 will have a specific height in its cargo bay.

 

If the caravan is at an angle due to the loading ramp,  the height is H cos x i.e. always less than H.

If you want to reduce the ramp angle,   you put the caravan on a low-loader trailer and adjust the trailer height.

 

I guess that these existed in 1974.    And would be available at most places in Australia.

You probably would not want to fart about with ramps, trailers, ... in a war zone.    But your natives should be friendly.

 

David.

Obviously I have explained it poorly Neil. Before the caravan levels out while being pushed up the ramp, its wheels are still on the ramp and the leading edge of the van (actually the rear of the van) entering the cargo hold continues upward on the same gradient as the loading ramp... ever upwards. Should I do a sketch?

Ross McKenzie ValuSoft Melbourne Australia

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

My point was that you can minimise the ramp angle.    Obviously more complicated than using the ramp directly to the runway.

 

Better than wasting 6 months' hard work.

 

David.

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Back when I was a very small engineer, just starting work, I came across a herd of my elders and betters scratching their head over a card full of chunky components that wasn't doing what it ought to on a new amplifier. I stuck my nose in, pointed at a resistor, and asked 'is that the right value?' Turns out it wasn't - yellow-purple-red instead of yellow-purple-orange... an easy mistake to make. But I had seen that resistor - on those old paxolin boards full of 1/4W resistors and sprayed with wax - and it was shinier than the rest.

 

Neil

  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

It's quite possible that the C-130 guys, for whatever reason(s), really don't like using a different ramp angle.  Perhaps the hinge is only load-bearing at that specified angle, for example.

 

I've found a few, from the guy furiously tearing apart a customer's machine only to find he'd plugged the monitor into the wrong port (and the p/s obediently turned itself off without frying anything), the guy who's mouse needed packing material removed, and a few of mine I'm less proud of.  However, it's always worth noting that any error of mine is a "Trivial oversight" and any error of someone else's is a "Colossal blunder".   cheeky  S.