+
+
-
- Sebastian Götte
- c/o Praxis Dr. Götte
- Krankenhausstr. 1a
- 54634 Bitburg
- imprint@jaseg.net
-
-
-
-
-Haftungsbeschränkung für Inhalte der Website
---------------------------------------------
-
-Gemäß § 7 Abs. 1 TMG ist der Verantwortliche der Website i. S. v. § 5 TMG für eigene Informationen, die er zur Nutzung
-bereithält, nach den allgemeinen Gesetzen verantwortlich.
-
-Der Verantwortliche der vorbezeichneten Website übernimmt keine Haftung auf Aktualität, Richtigkeit und Vollständigkeit
-der auf dieser Website zur Verfügung gestellten Inhalte. Dies gilt nicht, wenn dem Verantwortlichen vorsätzliches oder
-grob fahrlässiges Verhalten vorzuwerfen ist. Die Inhalte wurden mit der größtmöglichen Sorgfalt erstellt. Dennoch kann
-die inhaltliche Richtigkeit insbesondere bei komplexen Themen nicht gewährleistet werden, so dass der Verantwortliche
-den Nutzern empfiehlt, bei wichtigen Informationen bei den zuständigen Stellen anzufragen oder rechtliche Beratung in
-Anspruch zu nehmen. Sofern kostenpflichtige Inhalte oder Dienste auf der Website zur Verfügung gestellt werden, handelt
-es sich dabei um unverbindliche Invitatio ad offerendum, welche lediglich zur Abgabe eines Angebots durch den Nutzer
-aufrufen und selbst kein verbindliches Angebot darstellen.
-
-Gemäß §§ 8 ff. TMG ist der Website-Betreiber für fremde Inhalte, die er für einen Nutzer veröffentlicht, nicht
-verantwortlich, sofern
-
-* er keine Kenntnis von der rechtswidrigen Handlung oder der Information hat und ihm im Falle von
- Schadensersatzansprüchen auch keine Tatsachen oder Umstände bekannt sind, aus denen die rechtswidrige Handlung oder
- die Information offensichtlich wird, oder
-
-* er unverzüglich tätig geworden ist, um die Information zu entfernen oder den Zugang zu ihr zu sperren, sobald er
- diese Kenntnis erlangt hat.
-
-Erlangt der Website-Betreiber Kenntnis von solchen rechtswidrigen Inhalten, werden diese unverzüglich entfernt.
-
-Haftung für ausgehende Links
-----------------------------
-
-Der Website-Betreiber verlinkt von seiner Website auf fremde Websites. Durch diese sog. „Hyperlinks“ wird der Nutzer
-direkt auf die fremde Website geleitet. Der Website-Betreiber hat dabei keinerlei Einfluss auf die Informationen der
-fremden Website. Daher kann keine Haftung für die Aktualität, Richtigkeit und Vollständigkeit der Inhalte der fremden
-Website übernommen werden. Der Website-Betreiber versichert jedoch, dass ihm zum Zeitpunkt des Setzens der Verlinkung
-keinerlei rechtliche Verstöße bekannt waren und er die fremde Website im Rahmen des Zumutbaren geprüft hat.
-
-Erhält der Website-Betreiber Kenntnis von der Rechtswidrigkeit der verlinkten Inhalte, wird der entsprechende Link
-unverzüglich entfernt.
-
-Urheberrechte
--------------
-
-Die auf dieser Webseite veröffentlichten Inhalte unterliegen dem deutschen Urheberrecht.
-
-Als Urheber i. S. v. § 7 UrhG stehen dem Website-Betreiber die alleinigen und ausschließlichen Verwertungsrechte gemäß
-§§ 15 ff. UrhG zu. Eine Vervielfältigung oder Verwendung sämtlicher Inhalte der Website in fremden elektronischen oder
-gedruckten Medien ist ohne vorherige Zustimmung des Website-Betreibers untersagt und wird ggf. verfolgt.
-
-Datenschutz
------------
-
-Die Website kann grundsätzlich ohne Eingabe von personenbezogenen Daten wie z. B. Name oder E-Mail-Adresse genutzt
-werden. Sollte die Möglichkeit der Eingabe solcher Daten bestehen, so erfolgt die Mitteilung dieser Daten auf
-freiwilliger Basis durch den Nutzer. Eine Weitergabe dieser Daten an Dritte ist ohne die ausdrückliche Zustimmung durch
-den Nutzer ausgeschlossen.
-
-Ein lückenloser Schutz der übermittelten Daten vor dem Zugriff durch unbefugte Dritte ist jedoch nicht möglich, da es im
-Bereich der Datenübermittlung im Internet zu Sicherheitslücken kommen kann. Der Website-Betreiber versucht jedoch, die
-Gefahr des unbefugten Zugriffs durch geeignete Maßnahmen zu unterbinden und im Falle der Kenntnis von einer
-Sicherheitslücke diese durch geeignete Maßnahmen zu schließen.
-
-Der Nutzung der im Impressum veröffentlichten Kontaktdaten durch Dritte zur Übersendung von nicht ausdrücklich
-gewünschter Werbung, insbesondere Spam-Mails, wird widersprochen und im Falle der Nichtbeachtung ggf. mit rechtlichen
-Schritten verfolgt.
-
-(Quelle: `BUSE HERZ GRUNST Rechtsanwälte 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
-
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
-
- 
- 
-
- 
- 
- 
- 
- 
- 
- 
- 
-
- 
-
- 
-
- 
- 
- 
- 
- 
- 
- 
- 
-
+Sebastian Götte
+c/o Praxis Dr. Götte
+Krankenhausstr. 1a
+54634 Bitburg
+imprint@jaseg.net
+
+Gemäß § 7 Abs. 1 TMG ist der Verantwortliche der Website i. S. v. § 5 TMG für eigene Informationen, die er zur Nutzung +bereithält, nach den allgemeinen Gesetzen verantwortlich.
+Der Verantwortliche der vorbezeichneten Website übernimmt keine Haftung auf Aktualität, Richtigkeit und Vollständigkeit +der auf dieser Website zur Verfügung gestellten Inhalte. Dies gilt nicht, wenn dem Verantwortlichen vorsätzliches oder +grob fahrlässiges Verhalten vorzuwerfen ist. Die Inhalte wurden mit der größtmöglichen Sorgfalt erstellt. Dennoch kann +die inhaltliche Richtigkeit insbesondere bei komplexen Themen nicht gewährleistet werden, so dass der Verantwortliche +den Nutzern empfiehlt, bei wichtigen Informationen bei den zuständigen Stellen anzufragen oder rechtliche Beratung in +Anspruch zu nehmen. Sofern kostenpflichtige Inhalte oder Dienste auf der Website zur Verfügung gestellt werden, handelt +es sich dabei um unverbindliche Invitatio ad offerendum, welche lediglich zur Abgabe eines Angebots durch den Nutzer +aufrufen und selbst kein verbindliches Angebot darstellen.
+Gemäß §§ 8 ff. TMG ist der Website-Betreiber für fremde Inhalte, die er für einen Nutzer veröffentlicht, nicht +verantwortlich, sofern
+Erlangt der Website-Betreiber Kenntnis von solchen rechtswidrigen Inhalten, werden diese unverzüglich entfernt.
+Der Website-Betreiber verlinkt von seiner Website auf fremde Websites. Durch diese sog. „Hyperlinks“ wird der Nutzer +direkt auf die fremde Website geleitet. Der Website-Betreiber hat dabei keinerlei Einfluss auf die Informationen der +fremden Website. Daher kann keine Haftung für die Aktualität, Richtigkeit und Vollständigkeit der Inhalte der fremden +Website übernommen werden. Der Website-Betreiber versichert jedoch, dass ihm zum Zeitpunkt des Setzens der Verlinkung +keinerlei rechtliche Verstöße bekannt waren und er die fremde Website im Rahmen des Zumutbaren geprüft hat.
+Erhält der Website-Betreiber Kenntnis von der Rechtswidrigkeit der verlinkten Inhalte, wird der entsprechende Link +unverzüglich entfernt.
+Die auf dieser Webseite veröffentlichten Inhalte unterliegen dem deutschen Urheberrecht.
+Als Urheber i. S. v. § 7 UrhG stehen dem Website-Betreiber die alleinigen und ausschließlichen Verwertungsrechte gemäß +§§ 15 ff. UrhG zu. Eine Vervielfältigung oder Verwendung sämtlicher Inhalte der Website in fremden elektronischen oder +gedruckten Medien ist ohne vorherige Zustimmung des Website-Betreibers untersagt und wird ggf. verfolgt.
+Die Website kann grundsätzlich ohne Eingabe von personenbezogenen Daten wie z. B. Name oder E-Mail-Adresse genutzt +werden. Sollte die Möglichkeit der Eingabe solcher Daten bestehen, so erfolgt die Mitteilung dieser Daten auf +freiwilliger Basis durch den Nutzer. Eine Weitergabe dieser Daten an Dritte ist ohne die ausdrückliche Zustimmung durch +den Nutzer ausgeschlossen.
+Ein lückenloser Schutz der übermittelten Daten vor dem Zugriff durch unbefugte Dritte ist jedoch nicht möglich, da es im +Bereich der Datenübermittlung im Internet zu Sicherheitslücken kommen kann. Der Website-Betreiber versucht jedoch, die +Gefahr des unbefugten Zugriffs durch geeignete Maßnahmen zu unterbinden und im Falle der Kenntnis von einer +Sicherheitslücke diese durch geeignete Maßnahmen zu schließen.
+Der Nutzung der im Impressum veröffentlichten Kontaktdaten durch Dritte zur Übersendung von nicht ausdrücklich +gewünschter Werbung, insbesondere Spam-Mails, wird widersprochen und im Falle der Nichtbeachtung ggf. mit rechtlichen +Schritten verfolgt.
+(Quelle: BUSE HERZ GRUNST Rechtsanwälte)
+On May 17 2019 I gave a short presentation on the fundamentals of hardware security modules at the weekly seminar of +Prof. Mori's security research working group at Waseda University. The motivation for this was that outside of low-level +hardware security people and people working in the financial industry HSMs are not thought about that often. In +particular most network or systems security people would not consider them an option. Also it could turn out to be +really interesting to think about what could be done with an HSM in conjunction with modern cryptography (instead of +just plain old RSA-OAEP and AES-CBC).
+ +Preparing for this talk brought me back to some research ideas I've been working on for a while now. Since I'm not sure +I'll find the time to properly research this topic, I thought it would be great to write down some rought outlines first +for future reference.
+Currently, HSMs are only used in certain specific niche applications such as certificate authority key management and +financial transaction data handling. One key reason for this is that HSMs currently don't provide the affordances that +would be needed for them to be adopted more widely by the cryptographic and security engineering community. As far as I +can tell, the two core missing affordances are:
+The first issue can be addressed by simply creating a viable low-cost alternative. There is no fundamental technical +reason for the high cost of HSMs. This cost is instead due to manufacturers trying to recoup their expenses for R&D as +well as certification from the small volumes HSMs are sold in.
+Compared to system integration and certification the pure R&D cost of HSM defense mechanisms themselves is not too high +in an academic context it should be feasible to develop a sort of HSM blueprint that can then be cheaply produced by +anyone in need. Since the application areas outlined here are far from the core business areas of the clients of +established HSM vendors this would most likely not be a realistic threat to any established vendor's business and a +co-existence of both should not pose any problems in the short term.
+Tackling the high cost of current HSM hardware with an open-source HSM blueprint would yield +several academic advantages beyond cost reduction.
+An academic HSM blueprint would need to be flexible so that researchers can adapt it to their particular problem. A +modular architecture would lend itself to this flexibility. Fundamentally, there would be three components to this +architecture. First, a base containing infrastructure such as the surveillance microcontroller, power supplies, +power supply filtering and hardware DPA countermeasures, and possibly a standardized mechanical and electrical +interface.
+Next to the base, a system integrator would put their payload. The nature of this payload is intentionally kept +unspecified, and it might be anything from a cryptographic microcontroller to a small embedded system such as a +raspberry pi single board computer. Keeping the payload open like this achieves two benefits: It gives the HSM +blueprint's user their familiar tooling and the hardware they need, allowing fast adoption. Someone well-versed in +e.g. Javascript could literally implement their cryptography in Javascript, run it on an off-the-shelf raspberry pi and +just apply the HSM blueprint around it. In addition, keeping the payload open reduces the scope of what needs to be +implemented. Building a general SDK on top of something like a bare ARM SoC such as a TI OMAP or a Freescale/NXP IMX +would be a considerable additional burden, on top of the actual HSM design. Keeping the payload open allows research +to concentrate on the actual point, the HSM design.
+The final and most important component would be a set of security measures that can be combined with the base to +form the final HSM. Each of these security measures would entail a detailed specification of its design, manufacture +and security properties. These security measures could be simple things like tamper switches or potting, but could +also be complex things like security meshes.
+Given these three components -- base, payload and security measures as detailed specifications any engineer should +be able to design and manufacture a HSM customized to their needs. Unifying these three components within the HSM +blueprint would be a set of reference designs. Each reference design would implement a particular parametrization of the +three architectural components with a physical hole cut out where the payload would go.. These reference designs would +for one serve to guide any implementer on the customization and integration of their own derivation from the blueprint. +In addition it would serve as an extremely simple, low-cost point of entry into the ecosystem. A curious researcher +could simply replicate the reference design and put their existing payload inside. Practically this might mean e.g. a +researcher having PCBs produced according to the design files for a reference design for a mesh-based HSM, producing +their own mesh, physically glueing a raspberry pi SBC into the middle of it, and potting the resulting system. Given the +ease of prototype PCB fabrication today this would realistically allow evaluation of HSM technologies on a budget that +is orders of magnitude less than the cost of current HSMs.
+The core component of an HSM blueprint would be a suite of tamper detection mechanisms. Following are a few ideas on how +to improve on the current state of the art of membrane tamper switches plus temperature sensors plus PCB and printed +security meshes plus potting.
+Advanced analog sensing +Self-test functionality
+Analog sensing
+3D metal patterning techniques refers to any technique for producing thin, patterned metal structures on a +three-dimensional plastic substrate. The basic process would consist of 3D-printing the polymer substrate, depositing a +thin metal layer on top and then patterning this metal layer. A good starting point here would be the recent work of +Ben Kraznow on this exact thing.
+Copper filament methods would be any method embedding copper wire from a spool in some resin or other matrix. This +could mean either of a systematic approach of carefully winding or folding the copper wire into patterns or a +non-systematic approach of simply stuffing a large tangle of copper wire into a small space. The main challenge with the +former would be to find a non-tedious way of production. The main challenge with the latter would be to find process +parameters that guarantee complete coverage of the HSM without holes or other areas of lower sensitivity to intrusions. +Both approaches would require careful consideration of the overall design including the polymer resin supporting +structure to ensure sensitivity against attacks since copper wire is mechanically much stronger than the micrometre-thin +metal coatings used in patterning techniques.
+Finally, I think there is another set of currently under-utilized tamper-detection methods that would be very +interesting to explore. I am not aware of an academic term for these, so I am just going to dub them envelope +measurement here.
+The fundamental apporach of a mesh is to build a physical security envelope (the mesh) that physically detects when it +is disturbed (open or short circuits). This approach works well but has the disadvantage that these meshes are rather +complex to manufacture since effectively every part of them is acting as a sensing element. A conceptually more complex +but in practice potentially simpler approach might be to split the functions of security envelope and sensing element. +This would mean that in place of the mesh, some form of passive element such as metal foil forms the security envelope +which is then checked for tampering using a very sensitive sensor inside. This remote-sensing approach might simplify +the manufacture of the envelope itself and thus yield a design that is more easily customized. Following are a few ideas +on how to approach this envelope measurement problem.
+Ultrasonic If the HSM is potted, a few ultrasonic transducers could be added inside the potting. With several +transducers, any one could be used to transmit ultrasound while the others measure complex phase and energy of the +signal they receive. The circuitry for this could be made fairly simple if using a static transmit frequency or a low +chirp rate by using a homodyne receiver built around a comparator fed into some timers. This approach would likely +detect any mechanical attack and would also rule out chemical attacks involving liquids (though starting from which +amount of liquid depends on receiver sensitivity). The main disadvantages might be high power consumption and cost and +size of the ultrasonic transducers. Traditional cheap transducers made for air as a transmission medium are fairly large +and might not adequately couple into potting compound. If somehow one could convince a standard small piezo element to +do the same job that would be great as far as cost and size are concerned. A concern in some fringe use cases might be +suceptibility to ambient noise, though this could easily be reduced at the expense of space and heat dissipation +capacity by adding sound dampening on the outside. A likely attack vector against this approach might be using a laser +cutter to drill a hole through the potting compound, then inserting probes carefully chosen to not couple too much +to the potting compound ultrasonically.
+Light In either an unpotted HSM or one potted with a transparent (at some wavelengths) potting compound one could +embed LEDs and photodiodes in a similar setup to the ultrasonic setup described above. In contrast to the ultrasound, +the LEDs would literally have to light up the HSM's interior and shadows might be an issue since the HSM is likely some +flat rectangular shape. A possible solution to this would be to coat both the embedded payload and the lid with some +highly reflective paint such as some glossy silver paint or simple white paint. The basic approach might be as simple as +simply turning on several LEDs distributed throughout the HSM in turn and measuring amplitude at several photodetectors, +or as complex as doing a LIDAR-like phase measurement sweeping through a range of frequencies to determine not only +absorption but also phase/distance characteristics between emitter LED and detector photodiode. Using some high-gain TIA +along with a homodyne detector (lock-in amplifier) and changing emitter intensity, very precise measurements of both +absorption and phase might be possible, as might be measurements through almost opaque, diffuse potting compounds such +as a grey epoxide resin. The main disadvantages of this method would likely be the need to thoroughly light-proof the +entire HSM (likely by wrapping it in metal foil) and the potentially high cost of transmitter and receiver circuitry +(nice TIAs aren't cheap). To be effective against attacks using e.g. very fine drills and probes the system would likely +have to be very sensitive.
+Radar Finally, one could turn to standard radar techniques to fingerprint the inside of the HSM. The goal here would +be fingerprinting instead of mapping since only changes need to be detected. In this approach one could use homodyne +detection to improve sensitivity and reduce receiver complexity, and sweep frequencies similar to an FMCW radar (but +probably without exploiting the self-demodulation effect). Besides high cost, this approach has two disadvantages. +First, such a system would likely not go beyond 24GHz or maybe 40GHz due to component availability issues. Even at 40GHz +the wavelength in the potting compound would be in the order of magnitude of several millimeters. Fine intrusions using +some tool chosen to not interact too much with the EM field inside the HSM such as a heated ceramic needle or simply a +laser cutter might not be detectable using this approach. In any case, this system would certainly not be able to detect +small holes piercing the HSM enclosure. The HSM enclosure would have to be made into an RF shield, likely by using some +metal foil in it.
+Overall in the author's opinion these three techniques are most promising in order Light, Ultrasonic, Radar. Light +would prbably provide the best sensitivity at expense of some cost. Ultrasonic might be used in conjunction with light +to cover some additional angles since it is potentially very low-cost. Radar seems hard to engineer into a solution that +works reliably and also would likely be at least an order of magnitude more expensive than the other two technologies +while not providing better sensitivity.
+
+Cryptography is at the foundation of our modern, networked world. From email to card payment infrastructure in brick and +mortar stores, cryptographic keys secure almost every part of our digital lives againts cybercriminals or curious +surveillance capitalists. Without cryptography, many of the things we routinely do in our lives such as paying for +groceries with a credit card, messaging a friend on Signal or unlocking a car with its keyfob +would not be possible. The security of all of these systems in its core lies on the secrecy of cryptographic keys. +Systems differ in what kind of keys they use, how often these keys are replaced and the intricacies of the cryptographic +operations these keys fit into but all have in common that their security relies on keeping the keys secret.
+In practice, this secrecy has been implemented in many different ways. Mass-market software such as browsers or +messenger apps usually relies on some operating system facility to tell the computer "please keep this piece of memory +away from all other applications". While on desktop operating systems usually this does not provide much of a barrier +to other programs on the same computer, on modern mobile operating systems this approach is actually quite secure. +However, given sufficient resources no security is perfect. All of these systems can be compromised if the host +operating system is compromised sufficiently, and for organizations with considerable resources a market has sprung up +that offers turn-key solutions for all wiretapping needs.
+In some applications, this level of security has not been considered sufficient. Particularly financial infrastructure +is such a high-profile target that a lot of effort has been put into the security of cryptographic implementations. The +best cryptographic algorithm is useless if it is run on a compromised system (from that system's point of view anyway). +One of the core cryptographic components in financial applications are smartcards like they are used as payment cards in +most countries nowadays. These smartcards contain a small, specialized cryptographic microcontroller that is designed to +be hard to tamper with. Though one of the design goals of the system is to reduce the amount of sensitive information +stored on the card, things such as copying of a card can only be hindered by making the chip hard to read out.
+
+With smartcards being the means of choice on one side of the counter in electronic payments, on the other side of the +counter a different technology prevails. Attacks on payment terminals are bound to have much more dire consequences than +attacks on individual cards since one terminal might see hundreds of cards being read every day. For this reason, the +level of attack countermeasures employed in these terminals is a considerable step up from bare smartcards. While a +smartcard is made physically hard to tamper, it does not have a battery and it can only detect tampering once it is +powered by a reader. This allows for well-equipped attackers to use tools such as Focused Ion Beam (FIB) workstations to +circumvent the smartcard's defences while it is powered down, and then power up the card to carry out the actual attack.
+The answer to this problem in electronic payment infrastructure is called Hardware Security Module, or HSM. An HSM is +similar to a smartcard in its function (cryptographic processing using keys that are meant to never leave the protection +of the HSM). The one major between the two is that an HSM has its own battery and is continuously powered from its +manufacture to the day it is scrapped. If the HSM looses power at any point in time, it uses a small amount of energy +stored internally to securely wipe all cryptographic secrets from its memory within a few milliseconds.
+Being powered at all times allows the HSM to actively detect and respond to attacks. The most common way this is done is +by wrapping the juicy secret parts in a foil or a printed circuit board that is patterned with a long and convoluted +maze of wires, called a mesh. The HSM is continuously monitoring these wires for changes (such as shorts, breaks or +changes in resistance) and will sound the alarm when any are detected. Practically, this presents a considerable hurdle +to any attacker: They have to find a way to disable or circumvent the mesh while it is being monitored by the HSM. In +practice, often this is no insurmountable challenge but it again increases attack costs.
+Throughout my studies in security research I have always had an interest in HSMs. I have taken apart my fair share of +HSMs and at this point, to understand the technology more, I want to experiment with building my own HSM. In last year's +HSM basics post I have lined out some ideas for a next generation design that +deviates from the bread-and-butter apporoach of using a mesh as the primary security feature. Before embarking on +practical experiments with these ideas, I want to first take a stab at replicating the current state of the art as best +I can. State of the art meshes often use exotic substrates such as 3D plastic parts with traces chemically deposited on +their surface or special flexible substrates with conductive ink traces. These technologies will likely be too +cumbersome for me to implement myself only for a few prototypes, and industrial manufacturers will most likely be too +expensive. Thus, I will concentrate on regular PCB technology for now.
+The idea of a mesh on a PCB is pretty simple: You have one or several traces that you try to cover every corner of the +mesh PCB's area with. To be most effective, the traces should be as thin and as close together as possible. To increase +the chances of a manipulation being detected, multiple traces can also be used that can then be monitored for shorts +between them.
+While one can feasibly lay out these traces by hand, this really is an ideal application of a simple auto-router. While +general PCB autorouting is hard, auto-routing just a few traces to approximate a space-filling curve is not. Since I +am just starting out, I went with the simplest algorithmic solution I could think of. I first approximate the area +designated to the mesh with a square grid whose cells are a multiple of my trace/space size. The mesh will only be drawn +into grid cells that are fully inside the set boundaries. All cells outside or going across the border are discarded in +this step.
+I decided to implement this auto-router in a KiCAD plugin. Though KiCADs plugin API is not the best, it was just about +usable for this task.
+
+
+ After generating the grid, starting from the place I want to connect to the mesh, I walk the grid's cells one by one to +generate a tree that covers the entire grid's area. To set the mesh's starting place I place a footprint on the board +(dark gray in the picture above) whose designator I then tell my script. The tree generation algorithm looks like a +depth-first search, except all checks are random. Depending on the level of randomness used at each step of the +algorithm it yields more or less organized-looking results. Below are five example runs of the algorithm at differing +levels of randomness with the cells colored according to their distance from the tree root. 0% randomness means that the +algorithm is going to try cells in forward direction first on every step, and only then try out left and right. 100% +means that on every step, the algorithm is choosing a new direction at random.
+
+
+
+
+
+ After I have built this tree like you would do in a depth-first search, I draw my one or several mesh mesh traces into +it. The core observation here is that there is only 16 possible ways a cell can be connected: It has four neighbors, +each of which it can either be connected to or not, which results in 2^4 options. If you consider rotations and +mirroring, this works out to rotations or mirrored versions of only six base tiles: The empty tile, a tile with all four +sides connected, a straight through, a 90 degree bend, and a "T"-junction—see the illustration below.
+
+ After tiling the grid according to the key above, we get the result below.
+
+
+ Putting it all together got me the KiCAD plugin you can see in the screenshot below.
+
+
+I am fairly happy with the result, but getting there was a medium pain. Especially KiCAD's plugin API is still very +unfinieshed. It is hard to use, most parts are completely undocumented and if you use anything but its most basic parts +things tend to break. One particular pain point for me was that after generating the mesh, the traces have been added to +the board, but are still invisible for some reason. You have to save the board first, then re-load the file for them to +become visible. Also KiCAD crashes whenever the plugin tries to remove a trace, so currently my workflow involves always +making a copy of the board file first and treating mesh generation as a non-reversible finishing step.
+ + +Recently, I have been working on a small driver for ambient lighting using 12V LED strips like you can get +inexpensively from China. I wanted to be able to just throw one of these somewhere, stick down some LED tape, hook it up +to a small transformer and be able to control it through Wifi. When I was writing the firmware, I noticed that when +fading between different colors, the colors look all wrong! This observation led me down a rabbit hole of color +perception and LED peculiarities.
+The idea of the LED driver was that it can be used either with up to eight single-color LED tapes or, much more +interesting, with up to two RGB or RGBW (red-green-blue-white) LED tapes. For ambient lighting high color resolution was +really important so you could dim it down a lot without flickering. I ended up using the same driver stage I used in the +multichannel LED driver project for its great color resolution and low hardware requirements.
+
+ To make setting colors over Wifi more intuitive I implemented support for HSV colors. RGB is fine for communication +between computers, but I think HSV is easier to work with when manually inputting colors from the command line. RGB is +close to how most monitors, cameras and the human visual apparatus work on a very low level but doesn't match +higher-level human color perception very well. When we describe a color we tend to think in terms of "hue" or +"brightness", and computing a measure of those from RGB values is not easy.
+Color spaces are a mathematical abstraction of the concept of color. When we say "RGB", most of the time we actually +mean sRGB, a standardized notion of how to map three numbers labelled "red", "green" and "blue" onto a perceived +color. HSV is an early attempt to more closely align these numbers with our perception. After HSV, a number of other +perceptual color spaces such as XYZ (CIE 1931) and CIE Lab/LCh were born, further improving this alignment. In +this mathematical model, mapping a color from one color space into another color space is just a coordinate +transformation.
+
+ CIE 1931 XYZ is much larger than any other color space, which is why it is a good basis to express other color spaces +in. In XYZ there are many coordinates that are outside of what the human eye can perceive. Below is an illustration of +the sRGB space within XYZ. The wireframe cube is (0,0,0) to (1,1,1) in XYZ. The colorful object in the middle is what +of sRGB fits inside XYZ, and the lines extending out from it indicate the space that can be expressed in sRGB but not in +XYZ. The fat white curve is a projection of the monochromatic spectral locus, that is the curve of points you get in +XYZ for pure visible wavelengths.
+As you can see, sRGB is much smaller than XYZ or even the part within the monochromatic locus that we can perceive. In +particular in the blues and greens we loose a lot of colors to sRGB.
+The wrong colors I got when fading between colors were caused by this coordinate transformation being askew. Thinking +over the problem, there are several sources for imperfections:
+
+
+ These last two errors are tricky to compensate. What I needed for that was basically a model of the perceived colors +of the LED tape's color channels. A way of doing his is to record the spectra of all color channels and then evaluate +their respective XYZ coordinates. If all three channels are measured in one go with the same setup the relative +magnitudes of the channels in XYZ will be accurate.
+To map any color to the LEDs, the color's XYZ coordinates simply have to be mapped onto the linear coordinate system +produced by these three points within XYZ. LEDs are mostly linear in their luminous flux vs. current characteristic so +this model will be adequate. The spectral integrals mapping the channels' measured responses to XYZ need only be +calculated once and their results can be used as scaling factors thereafter.
+In order to compensate for the cheap LED tape's non-ideal performance I had to measure the LED's red, green and blue +channels' spectra. The obvious thing would be to go out and buy a spectrograph, or ask someone to borrow theirs. The +former is kind of expensive, and I did not want to wait two weeks for the thing to arrive. The latter I could probably +not do every time I got new LED tape. Thus the only choice was to build my own.
+Luckily, building your own spectrometer is really easy. The first thing you need is something that splits incident light +into its constituent wavelengths. In professional devices this is called the `monochromator`_, since it allows extraction +of small color bands from the spectrum. The second thing is some sort of optics that project the incident light onto a +screen behind the monochromator. In professional devices lenses or curved mirrors are used. In a simple homebrew job a +pinhole as you would use in a camera obscura does a remarkably nice job.
+For the monochromator component several things could be used. A prism would work, but I did not have any. The +alternative is a diffraction grating. Professional gratings are quite specialized pieces of equipment and thus +rather expensive. Luckily, there is a common household item that works almost as well: A regular CD or DVD. The +microscopic grooves that are used to record data in a CD or DVD work the same as the grooves in a professional +diffraction grating.
+From this starting point, a few seconds on my favorite search engine yielded an article by two researchers from the +National Science Museum in Tokyo providing a nice blueprint for a simple cardboard-and-DVD construction for use in +classrooms. I replicated their device using a DVD and it worked beautifully. Daylight and several types of small LEDs I +had around did show the expected spectra. Small red, yellow, green, and blue LEDs showed narrow spectra, daylight one +continuous broad one, and white LEDs a continuous broad one with a distinct bright spot in the blue part. The +single-color LED spectra are quite narrow since they are determined by the LED's semiconductor's band gap, which is +specific to the semiconductor used and is quite precise. White LEDs are in fact a blue LED chip covered with a so-called +phosphor. This phosphor is not elementary phosphorus but an anorganic compound that absorbs the LED chip's blue light +and re-emits a broader spectrum of more yellow-ish wavelengths instead. The final LED spectrum is a superposition of +both spectra, with some of the original blue light leaking through the phosphor mixing with the broadband yellow +spectrum of the phosphor.
+
+
+
+
+ Now that I had a spectrograph, I needed a somewhat predictable way of measuring the spectrum it gave me.
+Pointing a camera at the spectrograph would be the obvious thing to do. This produces pretty images but has one critical +flaw: I wanted to acquire quantitative measurements of brightness across the spectrum. Since I don't have a precise +technical datasheet specifying the spectral response of any of my cameras I can't compare the absolute brightness of +different colors on their pictures. Some other sensor was needed.
+
+ Looking around my lab, I found a bag of SFH2701 visible-light photodiodes. Their +datasheet includes their spectral response so I can compensate for that, allowing precise-ish absolute intensity +measurements. Just like LEDs, photodiodes are extremely linear across several orders of magnitude. The datasheet of the +classic BPW34 photodiode shows that this photodiode's light current is exactly proportional to illuminance over at +least three orders of magnitude. The SFH2701 datasheet does not include a similar graph but its performance will be +similar. The SFH2701 photodiodes I had at hand were perfect for the job compared to the vintage BPW34 since their +active sensing area is really small (0.6mm by 0.6mm) compared to the BPW34 (a whopping 3mm by 3mm). If I were to use a +BPW34 I would have to insert some small apterture in front of it so it does not catch too broad a part of the +spectrum at once. The SFH2701 is small enough that if I just point it at the projected spectrum directly I will +already get only a small part of the spectrum inside its 0.6mm active area.
+To convert the photodiode's tiny photocurrent into a measurable voltage I built another copy of the transimpedance +amplifier circuit I already used in the multichannel LED driver. A transimpedance amplifier is an +amplifiert that produces a large voltage from a small current. The weird name comes from the fact that it works kind of +like an amplified resistor (which can be generalized as an impedance electrically). Apply a current to a resistor and +you get a voltage. A transimpedance amplifiert does the same with the difference that its input always stays at 0V, +making it look like an ideal current sink to the connected current source.
+Transimpedance amplifiers are common in optoelectronics to convert small photocurrents to voltages. In this instance I +built a very simple circuit with a dampened transimpedance amplifier stage followed by a simple RC filter for noise +rejection and a regular non-inverting amplifier using another op-amp from the same chip to further boost the filtered +transimpedance amplifier output. I put all the passives setting amplifier response (the gain-setting resistors and the +filter resistor and capacitors) on a small removable adapter so I could easily change them if necessary. I put a small +trimpot on the virtual ground both amplifers use as a reference so I could trim that if necessary.
+
+ Following are pictures of the preamplifier board. The connectors on the top-left side are two copies of the analog +signal for the ADC and a small panel meter. The SMA connector is used as the photodiode input since coax cables are +generally low-leakage and have built-in shielding. The circuit is powered via the micro-USB connector and the analog +ground bias voltage can be adjusted using the trimpot.
+For easy replacement, all passives setting gain and frequency response are on a small, pluggable carrier PCB made from a +SMD-to-DIP adapter.
+Flying-wire construction is just fine for this low-frequency circuit. In a high-speed photodiode preamp, the +transimpedance amplifier circuit would be highly sensitive to stray capacitance, but we're not aiming at high speed +here.
+
+
+ Given a way to measure intensity what remains missing is a way to scan a single photodiode across the spectrum.
+A cheap linear stage can be found in any old CD or DVD drive. These drives use a small linear stage based on a +stepper-driven screw to move the laser unit radially. Removing the laser unit and connecting a leftover stepper driver +module I was left with a small linear stage with about 45 steps per cm without microstepping enabled. The driver I used +was an A4988 module that required at least 8V motor drive voltage. I used a small micro USB-input boost converter +module to generate a stable 10V supply for the motor driver, with the USB's 5V rail used as a logic supply for the motor +driver.
+The SFH2701 can easily be mounted to the linear stage using a small SMD breakout board glued in place with thin wires +connecting it to the transimpedance amplifier. The DVD drive linear stage is not very strong so it is important that +this wire does not put too much strain on it.
+Above the photodiode, I mounted a small piece of paper on the linear stage to be used as a projection screen to align +the linear stage in front of the spectrometer viewing window. A line on the screen paper points to the photodiode die in +parallel to the linear stage allowing precise alignment.
+The whole unit with photodiode preamplifier, linear stage, photodiode and stepper motor driver finally looks like this:
+
+ The projection of the spectrum can be adjusted by moving the light source relative to the entry slot and by moving +around the grating DVD.
+To capture a spectrum, first the light source has to be mounted near the spectrograph's entry slot. The LED tape I +tested I just taped face-down directly into it. Next, the grating DVD has to be adjusted to make sure the spectrum +covers a sensible part of the photodiode's path. Mostly, this boils down to adjusting the photodiode distance and height +to match the vertical extent and wiggling the grating DVD to adjust the projection's horizontal position.
+After the optics are set-up, the photodiode preamplifier has to be adjusted. In my experiments, most LED tape at 5GΩ +required a high-ish amplification. The goal in this step is to maximize the peak response of the preamp to be just +shy of its VCC rail to make best use of its dynamic range. To adjust the pre-amp, I took several very coarsely-spaced +measurements to give me an estimate of the peak while I did not yet know its precise location.
+Since stray daylight totally swamped out the weak projection of the LED's spectrum I shielded the entire setup with a +small box made of black cardboard and two black t-shirts on top. This shielding proved adequate for all my measurements +but I had to be careful not to accidentially move the DVD that was stuck into the spectrograph with the shielding +t-shirts.
+For capturing a single spectrum I wrote a small python script that will automatically move the stepper in adjustable +intervals and take two measurements at each point, one with the LED tape off that can be used for offset calibration and +one with the LED tape on. All measurements are stored in a sqlite database that can then be accesssed from other +scripts.
+I built a small script that shows the progress of the current run and an jupyter notebook for data analysis. The jupyter +notebook is capable of live-updating a graph with the in-progress spectrum's data. This was quite useful as a sanity +check for when I made some mistake easy to spot in the resulting data.
+After one color channel is captured, the LED tape has to be manually set to the next color and the next measurement can +begin.
+
+ Data analysis consists of three major steps: Offset- and stray light removal, wavelength and amplitude calibration and +color space mapping.
+The first task is to remove the offset caused by dark current as well as stray light of the LED's bright primary +reflection on the DVD. The LED is very bright and only a small part of its light gets reflected by the grating towards +the photodiode screen. The remaining part of the light is reflected onto the table in front of the DVD spectrograph. +Though I covered all of this with black cardboard, some of that light ultimately gets reflected onto the photodiode. +This causes a large offset, in particular in the blue part of the spectrum since in this part the photodiode is closest +to the spectrograph's opening.
+The composite offset can be approximated with a second-order polynomial that is fitted to all the data outside of the +main peak's area. Since at this point the wavelength of each data point is still unknown this is done with a rough first +estimate of the three colors' peaks' locations and widths.
+The photodiode's response is strongly wavelength-dependent. In particular in the blue band, the photodiode's sensitivity +gets very poor down to about 20% at the edge to ultraviolet. This effect is strong enough to move the apparent location +of the blue peak towards red.
+
+ The problem is that in order to remove this non-linearity, we would already have to know the wavelength of the measured +light. Since I don't, I settled for a two-step process. First, a coarse wavelength calibration is done relative to the +red peak and the short-wavelength edge of the blue peak. The photodiode measurements are then sensitivity-corrected +using this coarse measurement. Then all three channel peaks are measured in the resulting data and a fine wavelength +estimate is produced by a least-squares fit of a linear function. This fine estimate is then used for a second +sensitivity correction of all original measurements and the scale is changed from stepper motor step count to +wavelength in nanometers.
+
+ Finally, to achieve the objective of measuring the LED tape's channels' precise color coordinates the measured spetra +have to be matched against the color spaces' color matching functions. The color matching functions describe how +strong the color space's idealized standard observer would react to light at a particular wavelength. Going from a +measured spectrum to color coordinates XYZ works by integrating over the product of the measurement and each color +coordinate's color matching function.
+The result are three color coordinates X, Y and Z for each channel R, G and B yielding nine coordinates in total. When +written as a matrix conversion between XYZ color space and LED-RGB color space is as simple as multiplying that matrix +(or its inverse) and a vector from one of the color spaces.
+In XYZ space, the set of colors that can be produced with this LED tape is described by the parallelepiped spanned by +the three channel's XYZ vectors. In the following figures, you can see a three-dimensional model of the RGB LED's color +space (colorful) as well as sRGB (white) for comparison plotted within CIE 1931 XYZ. There is no natural map to scale +both so for this illustration the LED color space has been scaled to fit. These figures were made with blender and a few +lines of python. The blender project file including all settings and the python script to generate the color space +models can be found in the project repo.
+As you can see, the result is pretty disappointing. The LED's color space parallepiped is very narrow, which is because +the blue channel is much brighter than the other two channels. An easy fix for this is to scale-up the RGB space and +drop any values outside XYZ. The scaling factor is a trade-off between color space coverage and brightness. You can +produce the most colors when you clip all channels to brightness of the weakest channel (green in this case), but that +will make the result very dim. Scaling brightness like that stretches the RGB parallelepiped along its major axis. Up to +a point the number of possible colors (the gamut) increases at expense of maximum brightness. When the parallelepiped is +stretched far enought for all three channel vectors to be outside the 1,1,1 XYZ-cube, maximum brightness continues to +decrease but the gamut stays constant. I don't know a simple scientific way to solve this problem, so I just played +around with a couple of factors and settled on 2.5 as a reasonable compromise. Below is an illustration.
+In the end, the above measurements yield two matrices: One for mapping XYZ to RGB, and one for mapping RGB to XYZ. Of +the several versions of CIE XYZ I chose the CIE 1931 XYZ color space as a basis for the firmware because it is most +popular. Mapping a color coordinate in one color space to the other is as simple as performing nine floating-point +multiplications and six additions. Mapping Lab or Lch to RGB is done by first mapping Lab/Lch to XYZ, then XYZ to RGB. +Lab to XYZ is somewhat complex since it requires a floating-point power for gamma correction, but any self-respecting +libc will have one of those so this is still no problem. Lch also requires floating-point sine and cosine functions, but +these should still be no problem on most hardware.
+My implementation of these conversions in the ESP8266 firmware of my Wifi LED driver can be found on Github. You +can view the Jupyter notebook most of the analysis above here.
+Together, a friend and I outfitted the small staircase at Berlin's Chaos Computer Club with nice, shiny RGB-WW LED tape +for ambient lighting. This tape is like regular RGB tape but with an additional warm white channel, which makes for much +more natural pastels and whites. There are several variants of RGBW tape. Cheap ones have separate RGB and white LEDs, +which is fine for indirect lighting but does not work for direct lighting. Since we wanted to mount our tape in channels +at the front of the steps, we had to use the slightly more expensive variant with integrated RGBW LEDs. These are LEDs +in the 5050 (5.0mm by 5.0mm) form factor common with RGB LEDs that have a small section divided off for the white +channel. The red, green and blue LED chips sit together in the larger section covered with clear epoxy and the white +channel is made up from the usual blue LED inside a yellow phosphor in the smaller section.
+Since we wanted to light up all of 15 steps, and for greatest visual effect we would have liked to be able to control +each step individually we had to find a way to control 60 channels of LED tape with a reasonable amount of hardware.
+LED tape has integrated series resistors and runs off a fixed 12V or 24V constant-voltage supply. This means you don't +need a complex constant-current driver as you'd need with high-power LEDs. You can just hook up a section of LED tape +to a beefy MOSFET to control it. Traditionally, you would do Pulse Width Modulation (PWM) on the MOSFET's input to +control the LED tape's brightness.
+Pulse Width Modulation is a technique of controlling the brightness of a load such as an LED with a digital signal. +The basic idea is that if you turn the LED on and off much too fast for anyone to notice, you can control its power by +changing how long you turn it on versus how long you leave it off.
+PWM divides each second into a large number of periods. At the beginning of each period, you turn the LED on. After +that, you wait a certain time until you turn it off. Then, you wait for the next period to begin. The periods are always +the same length but you can set when you turn off the LED. If you turn it off right away, it's off almost all the time +and it looks like it's off to your eye. If you turn it off right at the end, it's on almost all the time and it looks +super bright to your eye. Now, if you turn it off halfway into the cycle, it's on half the time and it will look to your +eye as half as bright as before. This means that you can control the LED's brightness with only a digital signal and +good timing.
+
+ PWM works great if you have a dedicated PWM output on your microcontroller. It's extremely simple in both hardware and +software. Unfortunately for us, controlling 32 channels with PWM is not that easy. Cheap microcontrollers only have a +handful of hardware PWM outputs, so we'd either have to do everything in software, bit-banging our LED modulation, or +we'd have to use a dedicated chip.
+Doing PWM in software is both error-prone and slow. Since the maximum dynamic range of a PWM signal is limited by the +shortest duty cycle it can do, software PWM being slow means it has poor PWM resolution at maybe 8 bits at most. Poor +color resolution is not a problem if all you're doing is to fade around the HSV rainbow, but for ambient lighting +where you really want to control the brightness down to a faint shimmer you need all the color resolution you can get.
+If you rule out software PWM, what remains are dedicated hardware PWM controllers. Most of these have either of three +issues:
+This means we're stuck in a dilemma between two poor solutions if we'd want to do PWM. Luckily for us, PWM is not the +only modulation in town.
+PWM is the bread-and-butter of the maker crowd. Everyone and their cat is doing it and it works really well most of the +time. Unbeknownst to most of the maker crowd, there is however another popular modulation method that's mostly used in +professional LED systems: Enter *Binary Code Modulation* (BCM).
+BCM is to PWM sort of what barcodes are to handwriting. While PWM is easy to understand and simple to implement if all +you have is a counter and an IO pin, BCM is more complicated. On the other hand, computers can do complicated and BCM +really shines in multi-channel applications.
+Similar to PWM, BCM works by turning on and off the LED in short periods fast enough to make your eye perceive it as +partially on all the time. In PWM the channel's brightness is linearly dependent on its duty cycle, i.e. the percentage +it is turned on. In PWM the duty cycle D is the total period T divided by the on period T_on. The issue with doing PWM +on many channels at once is that you have to turn off each channel at the exact time to match its duty cycle. +Controlling many IO pins at once with precise timing is really hard to do in software.
+BCM avoids this by further dividing each period into smaller periods which we'll call bit periods and splitting each +channel's duty cycle into chunks the size of these bit periods. The amazingly elegant thing in BCM now is that as you +can guess from the name these bit periods are weighted in powers of two. Say the shortest bit period lasts 1 +microsecond. Then the second-shortest bit period is 2 microseconds and the third is 4, the fifth 8, the sixth 16 and so +on.
+
+ Staggered like this, you turn on the LED for integer value of microseconds by turning it on in the bit periods +corresponding to the binary bits of that value. If I want my LED to light for 19 microseconds every period, I turn it on +in the 16 microsecond bit period, the 2 microsecond bit period and the 1 microsecond bit period and leave it off for the +4 and 8 mircosecond bit periods.
+Now, how this is better instead of just more complicated than plain old PWM might not be clear yet. But consider this: +Turning on and off a large number of channels, each at its own arbitrary time is hard because doing the timing in +software is hard. We can't use hardware timers since we only have two or three of those, and we have 32 channels. +However, we can use one hardware timer to trigger a really cheap external latch to turn on or off the 32 channels all at +once. With this setup, we can only controll all channels at once, but we can do so with very precise timing.
+All we need to do is to set our timer to the durations of the BCM bit periods, and we can get the same result as we'd +get with PWM with only one hardware timer and a bit of code that is not timing-critical anymore.
+BCM is a truly wondrous technique, and outside of hobbyist circles it is in fact very widely known. Though we're using +it to control just 32 channels here, you can do much more channels without any problems. The most common application +where BCM is invariably used is any kind of LED screen. Controlling the thousands and thousands of LEDs in an LED +screen with PWM with a dedicated timer for each LED would not be feasible. With BCM, all you need to dedicate to a +single LED is a flipflop (or part of one if you're multiplexing). In fact, there is a whole range of ICs with no other +purpose than to enable BCM on large LED matrices. Basically, these are a +high-speed shift register with latched outputs much like the venerable 74HC595, only their outputs are constant-current +sinks made so that you can directly connect an LED to them.
+In our case, we don't need any special driver chips to control our LED tape. We just connect the outputs of a 74HC595 +shift register to one MOSFET each, and then we directly connect the LED tape to these MOSFETs. The MOSFETs allow us to +drive a couple of amps into the LED tape from the weak outputs of the shift register.
+The BCM timing is done by hooking up two timer channels of our microcontroller to the shift registers strobe and +reset inputs. We set the timer to PWM mode so we can generate pulses with precise timing. At the beginning of each +bit period, a pulse will strobe the data for this bit period that we shifted in previously. At the end of the bit +period, one pulse will reset the shift register and one will strobe the freshly-reset zeros into the outputs.
+
+ Our implementation of this system runs on an STM32F030F4P6, the smallest, cheapest ARM microcontroller you can get from +ST. This microcontroller has only 16kB of flash and 1kB of RAM, but that's plenty for our use. We use its SPI controller +to feed the modulation data to the shift registers really fast, and we use two timer channels to control the shift +registers' reset and strobe.
+We can easily cascade shift registers without any ill side-effects, and even hundreds of channels should be no problem +for this setup. The only reason we chose to stick to a 32-channel board is the mechanics of it. We thought it would be +easier to have several small boards instead of having one huge board with loads of connectors and cables coming off it.
+The BOM cost per channel for our system is 3ct for a reasonable MOSFET, about 1ct for one eighth of a shift register +plus less than a cent for one resistor between shift register and MOSFET. In the end, the connectors are more expensive +than the driving circuitry.
+From this starting point, we made a very prototype-y hardware design for a 32-channel 12V LED tape driver. The design is +based on the STM32F030F4P6 driving the shift registers as explained above. The system is controlled through an RS485 +bus that is connected up to the microcontroller's UART using an MAX485-compatible RS485 transceiver. The LED tape is +connected using 9-pin SUB-D connectors since they are cheap and good enough for the small current of our short segments +of LED tape. The MOSFETs we use are small SOT-23 logic-level MOSFETs. In various prototypes we used both International +Rectifier's IRLML6244 as well as Alpha & Omega Semiconductor's AO3400. Both are good up to about 30V/5A. Since we're +only driving about 2m of LED tape per channel we're not going above about 0.5A and the MOSFETs don't even get warm.
+During testing of our initial prototype, we noticed that the brightness seemed to jump around when fading to very low +values. It turned out that our extremely simple LED driving circuit consisting of only the shift register directly +driving a MOSFET, which in turn directly drives the LED tape was maybe a little bit too simple. After some measurements +it turned out that we were looking at about 6Vpp of ringing on the driver's output voltage. The picture below is the +voltrage we saw on our oscilloscope on the LED tape.
+
+ A bit of LTSpice action later we found that the inductance of the few metres of cable leading to the LED tape is the +likely culprit. The figure below is the schematic used for the simulations.
+
+ As tested, the driver does not include any per-output smoothing so the ~.5A transient on each BCM cycle hits the cable +in full. Combined with the cable inductance, this works out to a considerable lag of the rising edge of the LED +current, and bad ringing on its falling edge. Below is the voltage on the LED output from an LTSpice simulation of our +driver.
+
+ We were able to reduce the rining and limit the effect somewhat by putting a 220Ω series resistor in between the shift +register output and the MOSFET gate. This resistor forms an RC circuit with the MOSFET's nanofarad or two of gate +capacitance. The result of this is that the LED current passing the wire's ESL rises slightly more slowly and thus the +series inductance gets excited slightly less, and the overshoot decreases. Below is a picture of the waveform with the +damping resistor in place and a picture of our measurement for comparison. The resistor values don't agree perfectly +since the estimated ESL and stray capacitance of the wiring is probably way off.
+
+
+ A side effect of this fix is that now the effective on-time of the LED tape is much longer than the duty cycle at the +shift register's output at very small duty cycles (1µs or less). This is caused by the MOSFET's miller +plateau. For illustration, below is a graph of both the excitation waveform (the boxy line) and the resulting LED +current (the other ones) both without damping (top) and with 220Ω damping (bottom). As you can see the effective duty +cycle of the LED current is not at all equal to the 50% duty cycle of the excitation square wave.
+
+
+ In conclusion, we have three major causes for our calculated LED brightness not matching reality:
+Added up, these three effects yield a picture that agrees well with our simulations and measurements. The overall effect +is neglegible at long period durations (>10µs), but gets really bad at short period durations (<1µs). The effect is +non-linear, so correcting for it is not as simple as adding an offset.
+In order to correct for the nonlinearities mentioned above, we decided to implement a lookup table mapping BCM period to +actual timer setting. That is, each row of the table contains the actual period length we need to set the +microcontroller's timer to in order to get our intended brightness steps.
+To calibrate our driver, we needed a setup for reproducible measurement of the relative brightness of our LED tape at +different settings. Absolute brightness is not of interest to us as the eye can't perceive it. To perform the +calibration, the LED driver is set to enable each single BCM period in turn, i.e. brightness values 1, 2, 4, 8, 16 etc.
+The setup we used to measure the LED tape's brightness consists of a bunch of LED tape stuck into a tin can for +shielding against both stray light and electromagnetic interference and a photodiode looking at the LED tape. We used +the venerable BPW34 photodiode in our setup as I had a bunch leftover from another project and because they are quite +sensitive owing to their physically large die area.
+
+ The photodiode's photocurrent is converted into a voltage using a very simple transimpedance amplifier based around a +MCP6002 opamp that was damped into oblivion with a couple nanofarads of capacitance in its feedback loop. The MCP6002 +is a fine choice here since I had a bunch and because it is a CMOS opamp, meaning it has low bias current that would +mess up our measurements. For many applications, opamp bias current is not a big issue but when using the opamp to +directly measure very small currents at its input it quickly swamps out the signal for most BJT-input types.
+The transimpedance amplifier's output is read from the computer using the ADC input of a buspirate USB thinggamajob. In +general I would not recommend the buspirate as a tool for this job since it's ADC is not particularly good and it's +programming interface is positively atrocious, but it was what I had and it beat first wiring up one of the dedicated +ADC chips I had in my parts bin.
+The computer runs a small python script cycling the LED tape through all its BCM period settings and taking a brightness +measurement at each step. Later on, these measurements can be plotted to visualize the resulting slope's linearity, and +we can even do a simulation of the resulting brightness for all possible control values by just adding the measured +photocurrents for a certain BCM setpoint just as our retinas would do.
+
+ While it would be possible to fully automate the optimization of BCM driver lookup tables, we needed only one and in the +end I just sat down and manually tweaked the ideal values we initially calculated until I liked the result. You can see +the resulting brightness curve below.
+
+
+ Now that our driver was behaving linear enough that you couldn't see it actually wasn't we needed a nice way to control +it from a computer of our choice. In the ultimate application (our staircase) we'll use a raspberry pi for this. Since +we already settled on an RS485 bus for its robustness and simplicity, we had to device a protocol to control the driver +over this bus. Here, we settled on a simple, COBS-based protocol for the reasons I wrote about in How to talk to your +microcontroller over serial.
+To address our driver nodes, we modified the Makefile to build a random 32-bit MAC into each firmware image. The +protocol has only five message types:
+Wrapped up in a nice python interface we can now easily enumerate any drivers we connect to a bus, query their status +and control their outputs.
+
+
+
+
+ Putting some thought into the control circuitry and software, you can easily control large numbers of channels of LEDs +using extremely inexpensive driving hardware without any compromises on dynamic range. The design we settled on can +drive 32 channels of LED tape with a dynamic range of 14bit at a BOM cost of below 10€. All it really takes is a couple +of shift registers and a mildly bored STM32 microcontroller.
+Get a PDF file of the schematic and PCB layout here or download the CAD files +and the firmware sources from github. You can view the Jupyter notebook used to +analyze the brightness measurement data here.
+Private Contact Discovery (PCD) is the formal name for the problem modern smartphone messenger applications have on +installation: Given a user's address book, find out which of their contacts also use the same messenger without the +messenger's servers learning anything about the user's address book. The widespread non-private way to do this is to +simply upload the user's address book to the app's operator's servers and do an SQL JOIN keyed on the phone number field +against the database of registered users. People have tried sprinkling some hashes over these phone numbers in an +attempt to improve privacy, but obviously running a brute-force preimage attack given a domain of maybe a few billion +valid inputs is not cryptographically hard.
+Private Contact Discovery can be phrased in terms of Private Set Intersection (PSI), the cryptographic problem of having +two parties holding one set each find the intersection of their sets without disclosing any other information. PSI has +been an active field of research for a while and already yielded useful results for some use cases. Alas, none of those +results were truly practical yet for usage in PCD in a typical messenger application. They would require too much CPU +time or too much data to be transferred.
+At USENIX Security 2019, Researchers from technical universities Graz and Darmstadt published a paper titled Private +Contact Discovery at Scale +(eprint | PDF). +In this paper, they basically optimize the hell out of existing cryptographic solutions to private contact discovery, +jumping from a still-impractical state of the art right to practicality. Their scheme allows a client with 1k contacts +to run PCD against a server with 1B contacts in about 3s on a phone. The main disadvantage of their scheme is that it +requires the client to in advance download a compressed database of all users, that clocks in at about 1GB for 1B users.
+I found this paper very interesting for its immediate practical applicability. As an excuse to dig into the topic some +more, I gave a short presentation at my university lab's research seminar on this paper +(slides: PDF | ODP).
+Even if you're not working on secure communication systems on a day-to-day basis this paper might interest you. If +you're working with social account information of any kind I can highly recommend giving it a look. Not only might your +users benefit from improved privacy, but your company might be able to avoid a bunch of data protection and +accountability issues by simply not producing as much sensitive data in the first place.
+Scroll to the end for the TL;DR.
+In this article I will give an overview on the protocols spoken on serial ports, highlighting common pitfalls. I will +summarize some points on how to design a serial protocol that is simple to implement and works reliably even under error +conditions.
+If you have done low-level microcontroller firmware you will regularly have had to stuff some data up a serial port to +another microcontroller or to a computer. In the age of USB, a serial port is still the simplest and quickest way to get +communication to a control computer up and running. Integrating a ten thousand-line USB stack into your firmware and +writing the necessary low-level drivers on the host side might take days. Poking a few registers to set up your UART to +talk to an external hardware USB to serial converter is a matter of minutes.
+This simplicity is treacherous, though. Oftentimes, you start writing your serial protocol as needs arise. Things might +start harmless with something like SET_LED ON\n, but unless you proceed it is easy to end up in a hot mess of command +modes, protocol states that breaks under stress. The ways in which serial protocols break are manifold. The simplest one +is that at some point a character is mangled, leading to both ends of the conversation ending up in misaligned protocol +states. With a fragile protocol, you might end up in a state that is hard to recover from. In extreme cases, this leads +to code such as this gem performing some sort of arcane ritual to get back to some known state, and all just because +someone did not do their homework. Below we'll embark on a journey through the lands of protocol design, exploring the +facets of this deceptively simple problem.
+The first serial protocol you've likely written is a human-readable, text-based one. Text-based protocols have the big +advantage that you can just print them on a terminal and you can immediately see what's happening. In most cases you can +even type out the protocol with your bare hands, meaning that you don't really need a debugging tool beyond a serial +console.
+However, text-based protocols also have a number of disadvantages. Depending on your application, these might not matter +and in many cases a text-based protocol is the most sensible solution. But then, in some cases they might and it's good +to know when you hit one of them.
+Generally, you won't be able to stuff much more than four or five bit of information down a serial port using a +human-readable protocol. In many cases you will get much less. If you have 10 commands that are only issued a couple +times a second nobody cares that you spend maybe ten bytes per command on nice, verbose strings such as SET LED. But +if you're trying to squeeze a half-kilobyte framebuffer down the line you might start to notice the difference between +hex and base-64 encoding, and a binary protocol might really be more up to the job.
+On the computer side of thing, with the whole phalanx of an operating system, the standard library of your programming +language of choice and for all intents and purposes unlimted CPU and memory resources to spare you can easily parse +anything spoken on a serial port in real time, even at a blazing fast full Megabaud. The microcontroller side however is +an entirely different beast. On a small microcontroller, printf alone will eat about half your flash. On most small +microcontrollers, you just won't get a regex library even though it would make parsing textual commands so much +simpler. Lacking these resources, you might end up hand-knitting a lot of low-level C code to do something seemingly +simple such as parsing set_channel (13, 1.1333)\n. These issues have to be taken into account in the protocol design +from the beginning. If you don't really need matching parentheses, don't use them.
+Say you have a SET_DISPLAY command. Now say your display can display four lines of text. The obvious approach to this +is probably the SMTP or HTTP way of sending SET_DISPLAY\nThis is line 1\nThis is line 2\n\n. This would certainly +work, but it is very fragile. With this protocol, you're in trouble if at any point the terminating second newline +character gets mangled (say, someone unplugs the cable, or the control computer reboots, or a cosmic ray hits something +and 0x10 '\n' turns into 0x50 'P').
+You might try to solve the problem of your protocol state machine tangling up with a timeout. "If I don't get a valid +command for more than 200ms I go back to default state." But consider the above example. Say, your control computer +sends a SET_DISPLAY command every 100ms. If in one of them the state machine tangles up, the parser hangs since the +timeout is never hit, a new line of text arriving every 100ms.
+You might also try to drop the second newline and using a convention such as SET_DISPLAY is followed by two lines of +text, then commands resume.". This works as long as your display contents never look like commands. If you are only ever +displaying the same three messages on a character LCD that might work, but if you're displaying binary framebuffer +data you've lost.
+Always use a single line of text to represent a single command. Don't do protocol states or modes where you can toggle +between different interpretations for a line. If you have to send human-readable text as part of a command (such as +SET_DISPLAY) escape it so it doesn't contain any newlines.
+This way, you keep your protocol state machine simple. If at any time your serial trips and flips a bit or looses a byte +your protocol will recover on the next newline character, returning to its base state.
+Printing a number in hexadecimal is a very tidy operation, even on the smalest 8-bit microcontrollers. In contrast, +printing decimal requires both division and remainder in a loop which might get annoyingly code- and time-intensive on +large numbers (say a 32-bit int) and small microcontrollers.
+If you have to send fractional values, consider their precision. Instead of sending a 12 bit ADC result as a 32-bit +float formatted like 0.176513671875 sending 0x2d3 and dividing by 4096 on the host might be more sensible. If you +really have to communicate big floats and you can't take the overhead of including both printf and scanf you can +use hexadecimal floating point, which is basically hex((int)foo) + "." + hex((int)(65536*(foo - (int)foo))) for four +digits. You can also just hex-encode the binary IEEE-754 representation of the float, sending hex(*(int *)&float). +Most programming languages will have a simple, built-in means to parse this sort of thing.
+If you have to send arbitrary strings, escape special characters. This not only has the advantage of yielding a robust +protocol: It also ensures you can actually see everything that's going on when debugging. The string "\r\n" is easy to +distinguish from "\n" while your terminal emulator might not care.
+The simplest encoding to use is the C-style backslash encoding. Host-side, most languages will have a built-in means of +escaping a string like that.
+For binary data, hex and base-64 are the most common encodings. Since hex is simpler to implement I'd go with it unless +I really need the 30% bandwidth improvement base-64 brings.
+In contrast to anything human-readable, binary protocols are generally more bandwidth-efficient and are easier to format +and parse. However, binary protocols come with their own version of the caveats we discussed for text-based protocols.
+The most basic problems with binary protocols as with text-based ones is framing, i.e. splitting up the continuous +serial data stream into discrete packets. The issue is that it is that you have to somehow mark boundaries between +frames. The simplest way would be to use some special character to delimit frames, but then any 8-bit character you +could choose could also occur within a frame.
+Some protocols solve this problem much like we have solved it above for strings in line-based protocols, by escaping any +occurence of the special delimiter character within frames. That is, if you want to use 0x00 as a delimiter, you would +encode a packet containing 0xde 0xad 0x00 0xbe 0xef as something like 0xde 0xad 0x01 0x02 0xbe 0xef, replacing the +null byte with a magic sequence. This framing works, but is has one critical disadvantage: The length of the resulting +escaped data is dependent on the raw data, and in the worst case twice as long. In a raw packet consisting entirely of +null bytes, every byte must be escaped with two escape bytes. This means that in this case the packet length doubles, +and in this particular case we're even less efficient than base-64.
+Highly variable packet length is also bad since it makes it very hard to make any timing guarantees for our protocol.
+A framing mode sometimes used is to configure the UARTs to transmit 9-bit characters and to use the 9th bit to designate +control characters. This works really well, and gives plenty of control characters to work with. The main problem with +this is that a 9-bit serial interface is highly nonstandard and you need UARTs on both ends that actually support this +mode. Another issue is that though more efficient than both delmitier-based and purely text-based protocols, it still +incurs an extra about 10% of bandwidth overhead. This is not a lot if all you're sending is a little command every now +and then, but if you're trying to push large amounts of data through your serial it's still bad.
+Given the limitations of the two above-mentioned framing formats, we really want something better. The Serial Line +Internet Protocol (SLIP) as well as the Point to Point Protocol (PPP), standardized in 1988 and 1994 respectively, +both use escape sequences. This might come as a surprise, but humanity has actually still made significant technological +progress on protocols for 8-bit serial interfaces until the turn of the millennium. In 1999, Consistent Overhead Byte +Stuffing (COBS) (wiki) was published by a few +researchers from Apple Computer and Stanford University. As a reaction on the bandwidth doubling problem present in +PPP, COBS always has an overhead of a single byte, no matter what or how long a packet's content is.
+COBS uses the null byte as a delimiter interleaves all the raw packet data and a run-length encoding of the non-zero +portions of the raw packet. That is, it prepends the number of bytes until the first zero byte to the packet, plus one. +Then it takes all the leading non-zero bytes of the packet, unmodified. Then, it again encodes the distance from the +first zero to the second zero, plus one. And then it takes the second non-zero run of bytes unmodified. And so on. At +the end, the packet is terminated with a zero byte.
+The result of this scheme is that the encoded packet does not contain any zero bytes, as every zero byte has been +replaced with the number of bytes until the next zero byte, plus one, and that can't be zero. Both formatter and parser +each have to keep a counter running to keep track of the distances between zero bytes. The first byte of the packet +initializes that counter and is dropped by the parser. After that, every encoded byte received results in one raw byte +parsed.
+While this might sound more complicated than the escaping explained above, the gains in predictability and efficiency +are worth it. An implementation of encoder and decoder should each be about ten lines of C or two lines of Python. A +minor asymmetry of the protocol is that while decoding can be done in-place, encoding either needs two passes or you +need to scan forward for the next null byte.
+In binary protocols even more than in textual ones it is tempting to build complex state machines triggering actions on +a sequence of protocol packets. Please resist that temptation. As with textual protocols keeping the protocol state to +the minimum possible allows for a self-synchronizing protocol. A serial protocol should be designed such that if due to +a dropped packet or two both ends will naturally re-synchronize within another packet or two. A simple way of doing that +is to always transmit one semantic command per packet and to design these commands in the most idempotent way possible. +For example, when filling a framebuffer piece by piece, include the offset in each piece instead of keeping track of it +on the receiving side.
+Here's your five-step guide to serial bliss:
+In case you were having an inferiority complex because your friends' IBM Model M keyboards are so much louder than the +shitty rubber dome freebie you got with your pc... Here's the solution: Thor's Hammer, a simple typing cadence enhancer +for PS/2 keyboards.
+The connects to the keyboard's PS/2 clock line and briefly actuates a large solenoid on each key press. An interesting +fact about PS/2 is that the clock line is only active as long as either the host computer or the input device actually +want to send data. In case of a keyboard that's the case when a key is pressed or when the host changes the keyboard's +LED state, otherwise the clock line is silent. We ignore the LED activity for now as it's generally coupled to key +presses. By just triggering an NE555 configured as astable flipflop we can stretch each train of clock pulses to a +pulse a few tens of milliseconds long that is enough to actuate the solenoid.
+
+ Since PS/2 sends each key press and key release separately this circuit will pulse twice per keystroke. It would be +possible to ignore one of them but I figure the added noise just adds to the experience.
+Built on a breadboard, the circuit looks like this.
+
+ Since my solenoid did not have a tensioning spring I used a rubber band and some vinyl tape to make an adjustable +tensioner. The small orange USB hub serves as an end-stop because I had nothing else of the right shape. The sound and +resonance of the thing can be adjusted to taste by moving the end stop, adjusting the tensioning rubber and tuning the +excitation duration using the potentiometer. My particular solenoid was a bit slow so I added some pieces of circuit +board as shims between the plunger and the case to limit the plunger's travel inside the solenoid core.
+
+ After the multichannel LED driver was completed, I was just getting used to controlling LEDs at 14-bit resolution. +I liked the board we designed in this project, but at 32 channels it was a bit large for most use cases. Sometimes I +just want to pop a piece of LED tape or two somewhere, but I don't need a full 32 channels of control. I ended up +thinking that a smaller version of the 32-channel driver that didn't require a separate control computer would be +handy. So I sat down and designed a variant of the design with only 8 channels instead of 32 and an on-board ESP8266 +module instead of the RS485 transceiver for WiFi connectivity.
+The schematic was mostly copy-pasted from the 32-channel design. The PCB was designed from scratch. This time, I went +for a 5x7cm form factor to allow for enough room for all connectors and to give the ESP8266's WiFi antenna enough +space. The board has two 5-pin Phoenix-style for two RGB-White (RGBW) tapes and one 2-pin Phoenix-style connector for +12V power input. The control circuitry and the serial protocol are unchanged, but the STM32 now talks to an ESP-01 +module running custom firmware.
+The LEDs are driven using a 74HC595 shift register controlling a bunch of AO3400 MOSFETs, with resistors in front of +the MOSFETs' gates to slow down the transitions a bit to reduce brighntess nonlinearities and EMI resulting from +ringing of the LED tape's wiring inductance.
+The board has two spots for either self-resettable fuses (polyfuses) or regular melting-wire fuses in +a small SMD package, one for each RGBW output. For low currents the self-resettable fuses should be okay but at higher +currents their trip times get long enough that they become unlikely to trip in time to save anything, so plain old non-resettable fuses would be the way to go there.
+ + +
+
+
+ The STM32 firmware only had to be slightly modified to accomodate the reduced channel count since the protocol remains +unchanged. The ESP firmware is based on esphttpd by Spritetm. The modifications to the webserver firmware are pretty +basic. First, the UART console has been disabled since I use the UART to talk to the STM32. The few bootloader messages +popping out the UART on boot are not an issue, since they're unlikely to contain the fixed 32-bit address prefix the +serial protocol requires for the STM32 to do anything.
+Second, I added LED control by adding drivers for the serial protocol and a bunch of colorspace conversion functions. +When I first tested the prototype software, I noticed that color reproduction was extremely poor. When I just sent a +HSV rainbow fade from a python command line, the result looked totally wrong. The fade did not seem to go at a constant +speed and some colors, in particular yellow, orange and greens, were not visible at all. The problem turned out to be a +stark mismatch of the red, green and blue channels of the LED tape and less-than-optimal color reproduction of the pure +colors. I decided to properly measure the LED tape's color reproduction so I could compensate for it in software. This +turned out to be an extremely interesting project, the details of which you can read in my LED characterization +article.
+Third, I updated the built-in websites with some ad-hoc documentation on how to use the thing and a basic interface for +LED control.
+ +To be actually useful, the driver needed a robust enclosure. Bare PCBs are nice for prototyping, but for actually +putting the thing anywhere it needs a case to protect it against random destruction.
+The board has four mounting holes with comfortable spacing in its corners to allow easy mounting inside a 3D-printed +case. The case itself is described in an OpenSCAD script. To make it look a little nicer, a little 3D relief is laid +into the lid. The 3D relief is generated with a bit of blender magic. The source STL model is loaded into blender, then +blender's amazingly flexible rendering system is used to export a depth map of a projection of the model as a PNG file. +This depth map is then imported as a triangle mesh into OpenSCAD.
+For the relief to look good, I chose a rather high resolution for the depth map. This unfortunately leads to extreme +memory use and processing time on the part of OpenSCAD, but since I have access to a sufficiently fast machine that is +not a problem. Just be careful if you try opening the OpenSCAD file on your machine, OpenSCAD will probably crash +unless you're on a beefy machine or interrupt it when it starts auto-rendering the file.
+The board is mounted into the enclosure using knurled insert nuts that are pressed into a 3D-printed hole using a bit of +violence.
+ + + +
+
+ 
](https://github.com/goodroot/hugo-classic)
-
-# Hugo.io - Classic
-
-### For text focused goodness
-
-About twenty years ago Jobs and Wozniak, the founders of Apple, came up with the very strange idea of selling information processing machines for use in the home. The business took off, and its founders made a lot of money and received the credit they deserved for being daring visionaries. But around the same time, Bill Gates and Paul Allen came up with an idea even stranger and more fantastical: **selling computer operating systems**. This was much weirder than the idea of Jobs and Wozniak.
-
-~~~ruby
-def with_value_from_database(value)
- self.class.from_database(name, value, type)
-end
-
-def with_cast_value(value)
- self.class.with_cast_value(name, value, type)
-end
-end
-~~~
-
-A computer at least had some sort of physical reality to it. It came in a box, you could open it up and plug it in and watch lights blink. An operating system had no tangible incarnation at all. It arrived on a disk, of course, but the disk was, in effect, nothing more than the box that the OS came in.
-
-
