Electrical troubleshooting at work

[Adam10mm]

I don't normally do this, but my lab tech, my process engineer, and I are stumped.

We are monitoring a chemical bath using a controller that uses I think nanowatt pH probes to measure the pH of the bath and then dose replenishment chemistry as needed based on the pH. The problem is when a certain, single pump turns on to dose, it pulls the pH reading on the controller down and it bounces around by a full pH unit causing the pump to overdose and add too much chemistry.

We have tried those magnet things on the power cords to get rid of interference. We have three on the power cord and it's slightly tolerable.

We have replaced the pump with a brand new one of the same model and it was much worse at the problem.

What else do we need to look at to solve this problem? There is some kind of electrical or radio interference happening when just this pump is turning on. We can't get rid of it.

This is the pump model we are using, if anyone has experience with it. Thanks for any help you can offer. We are open to anything right now. Just trying to trouble shoot this.

http://www.ankoproducts.com/cased_913.html

[Lloyd Smale]

are you sure its not stray voltage due to a bad ground. Take a real sensitive volt meter and put one lead on your bench top and one to the neutral in the plug and see if you get a reading.

[OBIII]

I have no idea of the type/amount of power you are using to the pump in question, but how about an isolation transformer, or an inverter to create a stable voltage source?

OB

[dg31872]

Have no idea what you are talking about. However, have you considered it may not be electrical at all? What if the slug of replenishment chemistry circulates past the sensor which would cause the sensor to alter, causing swings in pH? I assume you have a mixing pump to stir that solution.
Now that I have shown my ignorance, I will shut up.

[Pb2au]

Grounding definitely could be an issue. Is the tank metal? If so, try and isolate it on a rubber pad.
Try and physically isolate the lead of the probe from everything.
When you installed the magnetic donuts on the power leads, did you coil the leads through it a few times? On our equipment that is our normal protocol.
When our company switched from hard wire connections of all of our low level inputs and outputs, to centralizing the connections to a CAN BUS system, we immediately learned hard lessons on necessary grounding of the entire machines and isolation of the power feeds.
Good luck. Just remember, don't let the magic smoke out of any electrical component. When something bad happens to an electrical component, the magic smoke will leak out and it will never work properly again.

[garym1a2]

If its a small pump try putting it on a computer UPS.
Another option is fine a seperate power outlet to run from.

[s mac]

I certainly am no electrical engineer, but as an auto tech I see wiring on vehicles sometimes wrapped in foil to lessen the effects of electrical interference, would be simple enough to try.

[country gent]

Could it be a "timing" the pump is pitting the replishment slug in faster than it is blended in or the sensors react to it? sort of like tricking into a digital scales.

[pull the trigger]

I was thinking temperature difference

[lavenatti]

Have no idea what you are talking about. However, have you considered it may not be electrical at all? What if the slug of replenishment chemistry circulates past the sensor which would cause the sensor to alter, causing swings in pH? I assume you have a mixing pump to stir that solution.
Now that I have shown my ignorance, I will shut up.

I am an Electrical Engineer and this was still my first thought.
Can you disconnect the probe signal during the addition (and for a second or two after) of pH balancing chemicals?
Might fix the problem whether it's electrical or chemical.

[Adam10mm]

No. The pH probe is continuously reading as the controller is constantly titrating in real time.

[Adam10mm]

Have no idea what you are talking about. However, have you considered it may not be electrical at all? What if the slug of replenishment chemistry circulates past the sensor which would cause the sensor to alter, causing swings in pH? I assume you have a mixing pump to stir that solution.
Now that I have shown my ignorance, I will shut up.
Not chemical. Chemical flow is not problematic. It happens when the dosing pump turns on to dose. The controller tells what pump to turn on to dose.

A different pump of this same exact model doesn't create the same problem when it turns on. When that pump is exchanged in the problem pump's place the same thing happens. It is something on that power line that is wacky.

I wish I could take pictures to show you but I am bound by ITAR.





Have no idea what you are talking about. However, have you considered it may not be electrical at all? What if the slug of replenishment chemistry circulates past the sensor which would cause the sensor to alter, causing swings in pH? I assume you have a mixing pump to stir that solution.
Now that I have shown my ignorance, I will shut up.

[tomme boy]

Are the wires for the probe and the wires for the pump run in the same conduit? If they are you need to separate them. To me it sounds like the probe is going out of tolerance. Do you have any type of mixer in the tank? Electric or air powered. That could also be putting out interferance. Also, if the tank is metal, run a seperate ground.

[catmandu]

I would guess that you may have a common mode intermod issue. The signal ground or common (from the PH probe) needs to be isolated from the supply ground. They make Isolators for this purpose. I assume (and may be wrong) that the probe output is 4-20 mA?

Paul

[Echo]

My guess, worth exactly what it costs, is that the power surge (drop) caused by the pump coming on confuses the probe circuitry. My suggestion is for a separate regulated power supply for the pumps, or maybe for the entire setup. SO I agree with OBIII, and garyM1A2.

[Adam10mm]

Are the wires for the probe and the wires for the pump run in the same conduit? If they are you need to separate them. To me it sounds like the probe is going out of tolerance. Do you have any type of mixer in the tank? Electric or air powered. That could also be putting out interferance. Also, if the tank is metal, run a seperate ground.

The probes in the controller are not with the pump power cord. The controller measures the pH in the controller by drawing a sample from the bath and turning the external pump on to dose.

There is a recirculation pump in the bath about 8 ft from the pump or controller. Electric powered pump. Tank is polymer. Shelf the pump is on is metal.

[Adam10mm]

I would guess that you may have a common mode intermod issue. The signal ground or common (from the PH probe) needs to be isolated from the supply ground. They make Isolators for this purpose. I assume (and may be wrong) that the probe output is 4-20 mA?

Paul

I will check the spec on the probes when I am done with lunch.

[Adam10mm]

The pumps and controller ( with probes) are on the same circuit I think. They are trying to go the cheapest route and I think they should be separate power supplies but that is above my pay grade.

[garym1a2]

Noise can be transmitted or conductive. Isolating the power supplies is a good start. So is sheilding the probes if possible since you are measuring milli-amps or less. When using my Oscilliscope to test Ignition systems I always use the ground on the scope tip to shield it. I also remove the ground on the power cord to the scope. In cases of high noise I add a big capacitor to the output of the power supply. If that does not work I run off battery power.

Sometimes you need to buy different equipment. Linear power suppliers instead of switchers, as an EE I find most if the time noise issues are solved by a capacitor in the right place.

[Cmm_3940]

I am an engineer. Take a close look at the equipment and the environment around it. Really, really close. Then ask yourself, "What has changed since it worked last?"

K.I.S.S.

As others have said, look at grounding and EMI. The thing is, if the system didn't need additional grounding/shielding/caps/ferrites yesterday, it shouldn't need them today.

Unless there is a new AM radio station in the neighborhood? :)

Wiggle all the connectors. For that matter, make sure they are actually connected. Oxidation happens. If it really is an electrical problem and it really did just start doing it, it is most likely something simple and stupid, which you can fix, or a bad chip/component, which you should leave to your engineer.

[bangerjim]

I am a process controls engineer by trade and have seen this many times in pH control loops large and small. It like chasing a ghost's tail! And very difficult to isolate and mediate.

Called a ground loop, you have some spurious current leakage in your system somewhere. Normally it is caused by metal contact in a pump or system with the liquid solution. But a peristaltic pump should have NO metal contact in it as the process liquid is squeezed thur the tubing in the plastic pump.

pH is a mv measurement at very high impedance and is susceptible to this. Somehow you are getting a current leakage from the pump controls. You said the problem does NOT travel with the pump, so that isolates it to that specific bench. I would investigate all the grounding wiring in all associated electrical equipment.

What brand/type of pH controller are you using? Does it use a preamp or does the probe hook up directly with an BNC connector (hi impedance connection)?

Titration probably is not the problem. Retention time in any pH controls system is critical, but sounds like your volume is pretty small, yet could be spiking the reaction. Do you have agitation?

And if agitation, that stirrer motor may be the ground loop culprit! And...check that recirc pump!

Good luck. I have fought systems like this all my professional (since 1972). It is not easy to isolate the offending equipment!

Pix sure would help but I have always had to sign non-disclosure statements when working in sites like yours.

banger

[Driver man]

I think switching the pump on introduces a transient voltage in the line which affects your metering process. In a high impedance circuit this effect can alter the hysteresis and null point of your probe electronics. I would check firstly all connecters for tight fitting then make sure cables are shielded and power and signal cables separated. Fit transient suppressors and isolating transformers if needed. You may need to scope your supply to actually see what is happening.

[crazy mark]

We use power conditioners for all our sensitive equipment as needed. I've seen too many pieces of equipment act up because of problems with small voltage spikes. The medical equipment usually is OK but certain office equipment has problems. We do have grounds in some parts of the building that is tied to all the metal structural components. A lot of computer servers in that building.

[Adam10mm]

Wiggle all the connectors. For that matter, make sure they are actually connected. Oxidation happens. If it really is an electrical problem and it really did just start doing it, it is most likely something simple and stupid, which you can fix, or a bad chip/component, which you should leave to your engineer.
This is with a new dosing controller that is more precise than the old controller. The old controller uses a colorimeter to measure the VLT (visible light transmission) of the bath and if component A is lower than spec, it turns all the dosing pumps on. We have no issues with this, but it's less precise control of the bath than the new controller. The new controller uses the titration and pH probes for two components and a colorimeter for a third component. It only turns on the dosing pump for the component that is needed at the time to bring the bath to the spec. That's where the problem is. With the old controller, it's less precise but no electrical issues. With the new controller, when it turns on the caustic pump, it causes the pH probes to drop in reading, rendering a false low and causing the dosing pumps to overshoot the set point (spec).

I KNOW it is something simple and cheap. That's the frustrating part we are all aware of. Complicated problems are caused by simple things. Guy was freaking out that the temperatures were over 70 degrees off spec; turns out he had the digital thermometer set to Celsius instead of Fahrenheit! :)

I'm just trying to help my process engineer troubleshoot this so we can assign a cause and have maintenance do what needs to be done to cure it. Sometimes it's helpful to have outside eyes and minds figure something out. It is frustrating not being able to post pictures of the setup, because I know that would help, but I don't want to get fired for releasing trade secrets and certainly not for ITAR violations (we are a government and military contractor) that could land me in prison.

Generally I stay in my lane as far as my engineer's territory (he is process control, I operate the line and execute that process) but he's even at a loss as to the cause and asked me however I can help, to please do it. I'm the machine operator but one of those guys that cares about what I do and the customers that are affected by what we make.


I am a process controls engineer by trade and have seen this many times in pH control loops large and small. It like chasing a ghost's tail! And very difficult to isolate and mediate.

Called a ground loop, you have some spurious current leakage in your system somewhere. Normally it is caused by metal contact in a pump or system with the liquid solution. But a peristaltic pump should have NO metal contact in it as the process liquid is squeezed thur the tubing in the plastic pump.

pH is a mv measurement at very high impedance and is susceptible to this. Somehow you are getting a current leakage from the pump controls. You said the problem does NOT travel with the pump, so that isolates it to that specific bench. I would investigate all the grounding wiring in all associated electrical equipment.
It's happened since we got the new controller a few months ago, but got progressively worse. There are 5 pumps on the shelf. 4 are dosing pumps and the other is a bailout pump. We dose chemistry into a bath tank but control the volume by bailing out excess volume. 2 of the dosing pumps are the same brand and type I listed in the original post. The other two are a different brand. This one pump and another one like it of the same brand/type are having this issue. Another pump on the shelf of this same brand/type is not having the issue.


What brand/type of pH controller are you using? Does it use a preamp or does the probe hook up directly with an BNC connector (hi impedance connection)?
I don't know the brand off hand or the probe connection. I do know the pH probes are grounded with the controller.


Titration probably is not the problem. Retention time in any pH controls system is critical, but sounds like your volume is pretty small, yet could be spiking the reaction. Do you have agitation?
The test jars in the controller have stir bars in each vessel (2 total). Volume of the bath or the test vessels? Bath volume is approximately 190 gallons. Test vessels less than 50 mL I would estimate.


And if agitation, that stirrer motor may be the ground loop culprit! And...check that recirc pump!
Agitation is pneumatic in the bath as well as an electric recirc pump.


Good luck. I have fought systems like this all my professional (since 1972). It is not easy to isolate the offending equipment!

Pix sure would help but I have always had to sign non-disclosure statements when working in sites like yours.

banger
It's frustrating having people that could help and solve the problem but me being bound by stuff that I can't release to the public. My engineer is waiting to hear back from the general electrician, but it seems the reason he is taking so long to get back to us to even look at it is he's stumped too.

[tomme boy]

Get rid of the electric recirc pump. That alone can induce current into the batch. Run a air powered one. Cheaper and easier to rebuild.

[MaryB]

Your caustic pump is putting some current on the liquid entering the tank, I remember the one I used at the feed testing lab was stainless and it screwed up the sensor that told me if the tank upstairs was full. We used the beat on the floor method to know when to turn it off.

[OBIII]

I just want to ask, that when the problem is resolved, please let us know what it was. Inquiring minds would like to know. Thanks.

OB

[MaryB]

One thing NOT to do is ground stuff at both ends. That will cause ground loops. Everything should have a common ground point

[RED333]

Get ya a few test leads with alligator clips on each end.
Use them to ground EVERYTHING to each other, every piece of
metal will need a ground. Them ground them to a good metal water
pipe that is in the dirt outside the building.
Now I know getting to a water pipe that is in dirt might be a bit tough,
but it will pull all the stray stuff away from you gear.

[375RUGER]

First thing I always check for is loose/broken wires.
Is this dosing motor AC or DC? I ask because you said the EMI filter helps, but you put them on the power cord. You may need filters on the motor power leads.
Does the power supply have sufficient capacity to power everything? Sounds to me like it is borderline.

[bangerjim]

This is NOT a solution an electrician can generally solve. This is an instrumentation problem related to electricity.

I would suspect your recirc equipment is causing a ground loop. Anywhere metal of any kind that comes in contact with your process can induce spurious electrical voltages into the liquid loop. Start eliminating them one by one until the problem is isolated. Just putting a ground wire with a clip on something may only create another loop on top of the one you have. Been there....done that!

The pH probe will be grounded via the BNC connector for noise rejection and as the return signal path for the mv signal from the probe. (pH is 49mv/pH unit, so it is a very low level signal at around 10Mohms impedance. You CANNOT read it with a digital VOM!) If you are using a separate temperature compensator sensor, which you really SHOULD, (normally 100 or 1000 ohm RTD), check that by disconnecting it. Sometimes that can lead to a ground loop problem if it has a metal case. That would be a defect in the new controller wiring you have.

That is what I have done for many years when something like this pops up. It is NOT an easy problem to rectify. (Try doing this on a 50,000 gallon REDOX 3 stage treatment system the size of a basketball court!) Again, I relate it to "chasing a ghost's tail"!

Good luck. The solution, once you find it, will be staring you right in the face!

Let us know.

bnagerjim

[Down South]

Pump Material.

[Houndog]

First off I AM NOT an electrical or instrument technician, but couldn't it be something as simple as dumping your doseing chemicals in the tank too fast? You stated you are overshooting your desired values, the new controller was more precise ( More exact in it's measurement) AND your PH probes are in a part of the tank that uses an agitator to move the product to the probes. Maybe you have had this problem all along and the more precise controller is picking up the readings your old system didn't. Lastly, is the end product being adversly affected or is this the results of lab tests only?

[C. Latch]

I'm not an engineer but when my engineering friends and clients start talking about stuff like this I listen and try to learn stuff, so I'm posting in this thread to make it easier to keep up with.

[bangerjim]

The advise I am giving here (based upon many years of college and the school of hard knocks) I usually charge my customers for on a consulting engineer basis of $120/hour. So enjoy the freebee.

banger

[leadman]

I read that the pump motor is 90v DC . There is a converter somewhere in the system that may be causing an electrical issue when the pump turns on. We had problems with these in some police vans for the DUI squad and they had to go to DC powered generators to solve the problems.
I also had to replace the convertor in my old motorhome when I installed a new TV. The old convertor had a mechanical switch on the transformer. The first time it opened it sent a spike thru the tv and fried it. I replaced it with a solid state unit.
I would check the electrical system for voltage drop at the pump compared to another pump. It can be as simple as a dirty or loose connection or corroded wires. You should also follow the wires to the sensor and see how they compare for reading to another unit.

[Adam10mm]

First off I AM NOT an electrical or instrument technician, but couldn't it be something as simple as dumping your doseing chemicals in the tank too fast?
Highly unlikely. Dosing rate cannot be changed either. It must stay where it is set.


You stated you are overshooting your desired values, the new controller was more precise ( More exact in it's measurement) AND your PH probes are in a part of the tank that uses an agitator to move the product to the probes.
pH probes are not in the tank, they are in small test vessels inside a controller. The test vessels have magnetic stir plates with stir bars to mix the sample during titrations. Based on those titrations in the controller, it will turn on the dosing pump to whichever component is needed. The old controller only sampled light and turned on all pumps when it needed to dose. The new controller only doses whatever component is needed.


Maybe you have had this problem all along and the more precise controller is picking up the readings your old system didn't. Lastly, is the end product being adversly affected or is this the results of lab tests only?
Controllers work very different ways. Old controller senses the color of the bath and turns all dosing pumps on if the color is low and it needs to dose. The new controller constantly titrates bath sample and samples color of the bath. Two components are measured for pH and one component for light. It only doses what component is needed.

Both end product and lab tests are being affected. Too much of two components leads to a "plate out" in which the metal component in the bath doesn't plate to the product and plates to the tank instead. Yes we are using DI water in the tank.

[popper]

Get a small cheap AM battery powered radio, tune to a weak station, turn on the pump ane listen for interference when the pump starts and stops. The specs dona't talk about a computer interface so I assume a relay operation. Run trumpet curves on the pump, I assume you replaced the tubing when checking with the other pump. Your controller/monitor - pump must be on different feeds. Check the pump power for slow over voltage on start of pump.
Edit: If there is a time delay in the new monitor (time lag), it will act like an open loop controller and cause improper dosing. The controller algorithm will need changing.

[C. Latch]

Get a small cheap AM battery powered radio, tune to a weak station, turn on the pump ane listen for interference when the pump starts and stops.

Hmmm.

OP, do you have a FLIR camera? You could look for loose (hot) connections. Might even be something tiny that you wouldn't think to look for.

[tomme boy]

The AM radio is used to check for interference. Poor mans O-scope. It works.

[tomme boy]

You aren't by chance phosphate coating are you? One of the places I worked at did light and heavy Phos coating of parts and when the nickle in the tank got too high it would start causing erroneous readings for the dose pumps.

[beagle]

Easy enough to EMI shield all your electrical leads/cables and tie the shields to ground using beryllium bonding straps. That way, you have no spare glitches running around. Sometimes, even the closing of switch contacts (contact bounce) can induce gremlins in the mix and the contacts have to be electronically conditioned. Check the aerospace industry for sources. I've been away from it for several years now and I don't know who's on top technology wise./beagle

[MaryB]

Never daisy chain your grounds, the last item in the chain can be at a different potential than the one at the ground end. Each piece should be grounded individually to a single point ground. My ham station has a ground buss bar that runs to my ground rod outside the house. Everything has its own ground lead to that buss bar.


Get ya a few test leads with alligator clips on each end.
Use them to ground EVERYTHING to each other, every piece of
metal will need a ground. Them ground them to a good metal water
pipe that is in the dirt outside the building.
Now I know getting to a water pipe that is in dirt might be a bit tough,
but it will pull all the stray stuff away from you gear.

[Adam10mm]

You aren't by chance phosphate coating are you? One of the places I worked at did light and heavy Phos coating of parts and when the nickle in the tank got too high it would start causing erroneous readings for the dose pumps.
Electroless copper. That's all I can say in public without losing my job and putting my family in financial turmoil.

[popper]

The pH probe is continuously reading - not if it is like the ones I used for H2O/O2 sensing. A few seconds delay.

[JonB_in_Glencoe]

I read that the pump motor is 90v DC . There is a converter somewhere in the system that may be causing an electrical issue when the pump turns on. We had problems with these in some police vans for the DUI squad and they had to go to DC powered generators to solve the problems.
I also had to replace the convertor in my old motorhome when I installed a new TV. The old convertor had a mechanical switch on the transformer. The first time it opened it sent a spike thru the tv and fried it. I replaced it with a solid state unit.
I would check the electrical system for voltage drop at the pump compared to another pump. It can be as simple as a dirty or loose connection or corroded wires. You should also follow the wires to the sensor and see how they compare for reading to another unit.

I don't know where Leadman got the 90VDC info. But if your pump is powered by a cheapo DC drive (back in the 90's, I was a tech at Extron, a DC drive manufacturer). They are noisey, Maybe you need to add a Isolation card to the input of the drive.
http://www.extroncompany.com/ProductandServices/IsolationCards/tabid/246/Default.aspx.html

[Adam10mm]

Problem solved. The pump turning on was drawing too much from the power supply. The pumps and the controller were on the same circuit. We plugged the controller into a different circuit and it was fine. Electrician installed a dedicated circuit for the controller to be powered. All is back to normal and we have been running a solid week without the problem ever occurring again. Thanks everyone for your help and suggestions.

[OBIII]

Glad to hear you got it fixed. Thanks for posting the info back here. I appreciate it.

OB

[waynem34]

somebody said O-scope.O-o

[MaryB]

Doesn't everybody have an O-scope? I have a tektronix 465 on the test bench.

[tomme boy]

Not many have an analog meter anymore either

[Mal Paso]

Doesn't everybody have an O-scope? I have a tektronix 465 on the test bench.

That's just plane cool! And no.

I do have a Triplett 630 though