Intro

Following on from Part 1 where we looked at a high-level teardown of a grid-tie inverter, here we do a deep dive into the internals - specifically the multi-CPU controller.

We’ll repeat the safety warning and then walk through the components on the CPU board, do some hardware reverse engineering, explain how to power it outside of the inverter away from dangerous voltages, extract some firmware and signals from the live board, do some protocol analysis and finally some firmware emulation.

Safety Warning (IMPORTANT)

Grid-tie inverters contain lethal voltages in many areas. Both high-voltage AC and DC are present on multiple internal rails, and disconnecting the unit does not automatically make it safe.

Large internal capacitors can retain 300–400 VDC for minutes after shutdown. There’s more than enough energy in those to kill you. These voltages are found on the MPPT input, the bulk DC bus, and the inverter H-bridge, and they’re spread across multiple PCBs.

If you haven’t worked with power electronics before, or you haven’t developed the instinct to spot something dangerous, this is not the project to start on.

This post documents what I found, how the inverter is structured, and what failed. It’s not how to repair your own unit. Read at your own risk!

Overview

The CPU board contains all the logic functions to control the Inverter’s operation. The operation is divided amongst three CPUs - a general purpose ATMega128 provides the user interface, both a human interface and an RS485 ModBus interface, and two specialised DSP processors for managing the MPPT boost converter and the inverter.

This article focuses on the UI CPU - we’ll tackle the other two in another post.

Identifying the components

The first step in understanding a new board is identifying the principal components. This was made more difficult than usual by the presnece of conformal coating (presumably how it gets to operate outside) which covered the part numbers making them very difficult to read. The coating was removed by carefully scratching it off with a guitar pick.

In the above diagram, the small squares, for example the one labelled “L2 primary relay” are the transistor drivers for the related component.

Powering the CPU and display boards outside of the inverter

The first power up attempt looked like this with 100 V from Makita tool batteries wired in series and the boost connected to the SMPS input directly.

As you can see it’s not very practical! The observant will notice the missing IGBT that was the cause of the initial fault - I snipped it out.

As I don’t feel like doing live probing with a high-current 100V supply, to make analysis more comfortable I decided to find out how to power the board without needing the whole inverter attached.

To do this, I first had to identify the pinout of the main header. This involved carefully tracing each connection using a multi-meter and photographs of both the CPU PCB and the Power Handling PCB to which it connects.

As there are 50 connections that took a little time! The result is shown in the figure to the right.

To power the board on the bench, you will need a power supply with the following voltages:

• 12 V
• -12 V (negative, this also works with -9 V)
• 16V (12 V also works for diagnostic purposes or you can leave disconnected if you just want to boot the CPUs.)

As my power supply does not produce negative voltages, I used 9 V PP3 battery with the positive terminal connected to ground to produce a negative voltage.

I measured the current drawn by the negative supply to be only 16 mA so it should last a reasonably long time if you don’t forget to disconnect it after testing!

The negative supply is used for both display contrast adjustment and elsewhere on the analogue side of the board. If it is not present, the display will not show properly, the Red LED illuminates and an error message is shown (faintly!).

The distinct 12 V IN points must both be connected.

If you power the board like that, you’ll get a promising boot followed quickly by:

The user manual for the inverter helpfully contains a list of the error codes and their meanings. For error 33, it lists it as “UnderTemperature.” I remembered seeing two temperature probes next to the IGBTs and boost MOSFETS so that became the next connector to reverse engineer. The result is as follows and shows the two temperature sensors RT1 and RT2 arriving on the connector.

Connecting those pins to +3V we have standalone boot success:

Firmware part 1: The ATMega CPU

After a little HRE with a multimeter and datasheets for the various inverting and tristate buffers, I traced the relevant functionality of the programming port for the ATMega128. Unlike the others, this port has an attached header so it’s likely the one used for factory programming:

The CPU port pins for programming are also used for other functionality on the board. To allow them to be shared with the programming port, the relevent pins are routed though tristate drivers whos function is selected by the nPDI_EN pin of the programming port. If you ground that pin, it makes the programming pins available for use.

Connecting a cheap AVR ISP (USPASP) programmer to the pins on this port enabled successful firmware download. The fuse bits preventing readout were not set.

Firmware emulation

I decided it would be fun to try to emulate the hardware and run the firmware on a PC. Using simavr (a fantastic AVR simulator), I loaded the extracted firmware and EEPROM images and was happy to discover it didn’t immediately crash!

Writing an emulation harness in C using simavr’s well designed API was very straight forward. I started off just dumping the traces to observe the behaviour, hoping to emulate the display.

The simavr package includes a hd44780 emulated peripheral so I wrote the C glue code to wire it up and dumped the display memory to the screen. IT WORKED so I quickly followed up with an ANSI layer to show the LEDs blinking and to keep the LCD display in one place.

But not for long. Another error this time Error 26, listed in the manual as Vref error. This lead me to beleive they’re sampling a vref somewhere, possibly using the ATMega’s built in ADC. I hooked up the ADC trigger IRQ and sure enough, the firmware is reading ADC0, ADC1 and ADC2. I measured the voltages arriving at those pins on my board and coded responses to the ADC trigger to provide those simulated voltages. A little further… It boots a bit more:

But after a while, we get Error 5, listed in the manual as a “Communications Error.” Thinking this is likely to do with comms to the other CPUs, I probed the various communications peripherals on the micro with an oscilloscope and discovered a continuous stream of data on the SPI peripheral. Using a second probe I was able to identify two chip selects, likely one for each micro. So I hooked up a logic analyser to the SPI and chip selects and made a recording.

Protocol analysis: just enough to boot, part 1

The logic analyser revealed two devices talking SPI, each using the same protocol.

I exported the raw bytes as binary from PulseView (the logic analyser GUI frontend), and loaded them into Python for analysis.

Part of coloured output is shown here, left is the outgoing (MOSI) packet, right is the incoming (MISO) packet:

The protocol appears to have 10-byte framing for requests and responses.

Master outgoing packet

The first 5 bytes of the master packet look like a possible write vector, i.e. a way to send data to the DSPs. The sixth byte looks like a register number. Finally the final four are dummy bytes to allow the slave device to respond.

Slave response packet

The response packet starts with what looks like a dummy byte (not shown). Followed by an echo of the first 5 bytes of the request. The sixth byte, on the other hand, is not echoed and looks to be always FF. Finally, the response is contained in the final 3 bytes.

The protocol reveals a large number of registers. I didn’t take the time to understand the 24-bit response format but perhaps it’s a custom 24-bit floating format used internally by the DSP.

Simulating just enough SPI.

Coding up the results of this analysis into the simulator, returning properly formatted packets with value of 0 for every register, allows the simulator to boot without error. Win! At this point I hooked up the simulated firmware’s buttons to keyboard keys and here’s the menu:

Next steps

We have a solid understanding of the Atmega128 CPUs place in this design. We are able even to emulate it with full UI interactability. Next up is a look at the DSP chips - can we do the same?