Debugging incorrect / malfunctioning keys (LEDs work)

[landakram]

I'm setting up KinT with a Kinesis Advantage 2 and a Teensy 3.6. I soldered everything and flashed the Teensy with this hex file.

When I plug everything in, the LEDs go on as expected, but keys are clearly malfunctioning: some keys aren't registering at all and some are registering as other keys or groups of keys (I'm using qmk configurator to test keypresses).

Any ideas on how to best to debug? I didn't do the best job soldering, but I'm not sure how to determine whether it's something there or software issue.

[stapelberg]

Hey, thanks for reaching out!

My first guess would be that some connection indeed isn’t soldered correctly.

Is there any rhyme or reason as to which keys in particular work / do not work?

That would help narrow it down.

[landakram]

OK, after getting hid_listen to compile on OSX and recompiling qmk with logging, it appears that column 3 (Shift, Z, X, C, V, B, etc. on qwerty) is the problem — every other column / row combination produces output except column 3 (3 function keys wired to TOP_RIGHT also have no output, presumably they are COL_3 as well).

I re-soldered all of the connections and then double-checked COL_3 connections, which as far as I understand from the schematic are:

• LEFT_13_LT pin 12
• RIGHT_13_DOWN pin 7
• TOP_RIGHT pin 8
• Teensy pin 20

But even after re-soldering these connections, COL_3 still doesn't produce any output 😭

I'm new to reading schematics; the FPC connector pins are staggered in 2 columns on the board but in the schematic they are vertical with 1 column. I want to make sure I'm looking at the right pins. Should I be reading them from the "1" on the silk screen such that pin 1 might be on the left column and then pin 2 might be up and to the right (and pin 3 up and to the left)? Or do I go up one column first?

Either way, I re-soldered the connections that result from both interpretations. But COL_3 still doesn't register... I am yet again at a loss and really appreciate any insights you might have!

Edit: After enabling more verbose debug options, I see the following when releasing a non-column-3 key:

r/c 01234567
00: 00010000
01: 00010000
02: 00010000
03: 00010000
04: 00010000
05: 00010000
06: 00010000
07: 00010000
08: 00010000
09: 00010000
0A: 00010000
0B: 00010000
0C: 00010000
0D: 00010000
0E: 00010000

So it appears that column 3 is "on" for all rows 🤔

[stapelberg]

I'm new to reading schematics; the FPC connector pins are staggered in 2 columns on the board but in the schematic they are vertical with 1 column. I want to make sure I'm looking at the right pins. Should I be reading them from the "1" on the silk screen such that pin 1 might be on the left column and then pin 2 might be up and to the right (and pin 3 up and to the left)? Or do I go up one column first?

Here’s how to get a better understanding of the connections:

1. Install kicad (free and open source)
2. Clone https://github.com/kinx-project/kint/ and open kicad/kint.pro in kicad
3. Select Tools → Edit PCB
4. Select View → Flip Board View, because the front side of kinT contains the LEDs, the back side contains the connectors.
5. Select Highlight Net, the second icon from the top in the right icon bar
6. Click on the pin of interest. In the bottom left, you’ll see the Net Name (e.g. COL_3), and kicad will highlight all connected traces.

For your specific case, this is the expected result:

Do you have a multimeter? You can use its connectivity test (beeps when there is a connection between both probe heads).

Do you have any left-over PCBs? Where did you make the PCBs?

Could you send photos of the soldering points for all the COL_3 connections as seen in the picture above?

The most common mistake really is a soldering mistake. Sometimes, residue can lead to unwanted connection. You could use a PCB cleaner if you haven’t already.

[Sanic]

Hi @landakram
Did you solve your problem in the meantime?
Can you please share what changes you had to made to hid_listen and how you debugged your problem further?
Since a lot of my keys also don't work on my board (even though they seem to be connected when checking with a multimeter) i have to dive into the debug options in QMK...

[stapelberg]

I don’t think you need any changes to hid-listen, you just need to start it and wait for debug messages to come in.

Which keys specifically don’t work for you? Compiling a list would be a good first step to narrow it down to specific rows or columns in the matrix, or to confirm that all rows/columns work fine per se and individual keys should be checked.

[landakram]

@Sanic, unfortunately I narrowed down my problem to a defective teensy pin (as far as I could tell) and then didn't go further.

I used the multimeter to test continuinity between every joint connecting COL_3 and the teensy pin. Every point was connected to the teensy pin.

I then used the multimeter to test voltage from the gpio pins. I found that every column pin had some non-zero voltage except for the affected pin, 23, which had a flat 0 volts. I concluded that pin 23 was defective, given that, from my understanding of the firmware, every column should be reading high when no keys are pressed. I neglected to test the pins before soldering everything, so perhaps the teensy was defective from the get-go or I fried it during my soldering troubles.

Can you please share what changes you had to made to hid_listen

I made some small changes to get it to compile for macOS-x86_64 using clang. The binary on the website is only compiled for powerpc. Please see the hid_listen.mac binary here, which should work for you on macOS: https://github.com/landakram/hid_listen/releases/tag/v1.0.0

@stapelberg, thank you for all your help. It was really helpful for a newbie like me. I hope to pick this project back up soon when I order another teensy 😄

[Sanic]

I don’t think you need any changes to hid-listen, you just need to start it and wait for debug messages to come in.

Which keys specifically don’t work for you? Compiling a list would be a good first step to narrow it down to specific rows or columns in the matrix, or to confirm that all rows/columns work fine per se and individual keys should be checked.

I did quick collection of the keys that don't work with the QMK Configurator:https://config.qmk.fm/#/test/
The result is in the following image. Everything that is marked with a red line is not working:

While testing the keys, i also found a weird behavior:
Pressing ',' or '.' doesn't work when you hit them alone.
However, if you hold ',' or '.' and THEN hit 'm', it also registered the ',' or '.'
/edit: I just noticed that this behavior also occurs on other areas on the keyboard, for example with CTRL on the right thumb cluster (which works) in conjunction with for example ENTER or SPACE (which don't work alone).
Any ideas on how to debug this further is highly appreciated 😃
I've built two redox keyboards before but haven't encountered such a behavior.

Ah btw: I've got a KB500 and also used a Teensy 3.6. I flashed the default .hex file from this github repo with the teensy flash tool.

/edit2:
When i connect the controller to my machine, i receive a single KEY_DOWN and a KEY_UP on Left Shift without pressing anything.
Does this also happen on your controller?

[Sanic]

As a follow up: I just tested the keyboard on a windows machine.
On that machine, it works flawlessly.

So i wonder what might cause these weird issues when using a Macbook.
I tried going through the Keyboard identification process and set the keyboard layout to ANSI and ISO, but that didn't change anything.
My redox keyboard also works without any problems on MacOS, so i wonder what might be different on the (ARM-based) firmware/hardware combination that could explain these problems.

[stapelberg]

This is really strange, but a good find!

I don’t have much time to look at this problem in detail right now, but here’s one thing to try: connect the keyboard via a USB hub. That way, subtle differences between USB implementations across devices/operating systems might be less of a problem.

[Sanic]

@stapelberg I also tried out to connect the keyboard via a Hub, but unfortunately that didn't change anything.

Today, i found the time to solder another board for the same keyboard, but with a teensy 2.0.
This board works perfectly on the same Mac.
So this issue might indeed be related to Teensy3.6 <-> MacOS.
I'll open another issue for that.

Thanks for your hints @stapelberg @landakram

[stapelberg]

I’ll close this issue as it seems there’s nothing else we need to diagnose/can do here. Please reopen if that’s incorrect.

[jsravn]

@landakram not sure you got back to this, but I went through the same fun exercise. The information here was helpful. In my case pin 14/15 were shorted, and pin 14 was shorted to GND. I'm unsure how this happened - there were no solder bridges at all, so I suspect my PCB was defective (next time, I'll check my PCB first).

I highly doubt the teensy pin itself is busted. I didn't want to junk my board and parts, so I hacked around it, maybe you can do something similar:

1. I pulled out the header pins from 14/15. Used solder wick to get as much as possible, then pliers to wiggle/pull them out.
2. After this, I verified there was no continuity between 14/15 and gnd directly on the teensy board.
3. I used a small knife to scratch out traces for pin 14 / 15 on the back.
4. Wired 14 and 15 each directly to their first respective FPC pin (see kicad for where they should connect - you can also see by following the traces) with some wire wrap wire.

Fortunately that was enough to get my board fully working. Hopefully you can get it working w/o ordering all new parts!