jaseg/blog
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

deploy.py auto-commit

Browse source
This commit is contained in:
jaseg 2025-10-21 22:10:36 +02:00
commit 2dbc148881
393 changed files with 7973 additions and 6787 deletions
Download patch file Download diff file Expand all files Collapse all files

1
.gitignore vendored
View file

@ -1 +0,0 @@
public

0
.gitmodules vendored
View file

119
about/index.html Normal file
View file

@ -0,0 +1,119 @@
<!DOCTYPE html>
<html><head>
<meta charset="utf-8">
<title>About jaseg | Home</title>
<meta name="description" content="">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="mobile-web-app-capable" content="yes">
<meta name="color-scheme" content="dark light">
<link rel="stylesheet" href="/style.css">
<link rel="preload" href="/fonts/roboto_slab/RobotoSlab-VariableFont_wght.ttf" as="font" type="font/ttf" crossorigin />
<link rel="preload" href="/fonts/nyght-serif-main/fonts/WEB/NyghtSerif-Regular.woff2" as="font" type="font/woff2" crossorigin />
<link rel="preload" href="/fonts/nyght-serif-main/fonts/WEB/NyghtSerif-Bold.woff2" as="font" type="font/woff2" crossorigin />
<link rel="preload" href="/fonts/nyght-serif-main/fonts/WEB/NyghtSerif-BoldItalic.woff2" as="font" type="font/woff2" crossorigin />
</head>
<body><nav>
<div class="internal">
<a href="/" title="Home">Home</a>
<a href="/blog/" title="Blog">Blog</a>
<a href="/projects/" title="Projects">Projects</a>
<a href="/about/" title="About" class="active">About</a>
</div>
<div class="search">
<div id="search"></div>
</div>
<div class="external">
<a href="https://git.jaseg.de/" title="cgit">cgit</a>
<a href="https://github.com/jaseg" title="Github">Github</a>
<a href="https://gitlab.com/neinseg" title="Gitlab">Gitlab</a>
<a href="https://chaos.social/@jaseg" title="Mastodon">Mastodon</a>
</span>
</nav>
<header>
<h1>About jaseg</h1>
<ul class="breadcrumbs">
<li><a href="/">jaseg.de</a></li><li><a href="/about/">About jaseg</a></li>
</ul>
</header>
<main data-pagefind-body>
<div class="document">
<div class="section" id="about">
<h2>About</h2>
<p>Hej, I'm Jan, or jaseg. At the moment I'm doing a PhD (Dr.-Ing.) at TU Darmstadt in Computer Science, specializing on
Hardware Security. This is my personal website where I publish things that I find interesting.</p>
<p>I self-host my code at <a class="reference external" href="https://git.jaseg.de/">git.jaseg.de</a>, but I am also on <a class="reference external" href="https://github.com/jaseg">github</a>
and on <a class="reference external" href="https://gitlab.com/neinseg">gitlab</a>. I use github for issue tracking for some of my projects such as
<a class="reference external" href="https://github.com/jaseg/gerbolyze">gerbolyze</a> and <a class="reference external" href="https://github.com/jaseg/python-mpv">python-mpv</a>. I maintain
the <a class="reference external" href="https://pypi.org/project/python-mpv/">python-mpv</a> and <a class="reference external" href="https://pypi.org/project/gerbolyze/">gerbolyze</a> python
packages on PyPI. Release tags on these two repositories are signed with the release signing key found <a class="reference external" href="https://github.com/jaseg.gpg">on github</a> and below.</p>
<p>I am not on any social network, but feel free to write me an email at <a class="reference external" href="mailto:hello&#64;jaseg.de?subject=About page on blog.jaseg.de">hello&#64;jaseg.de</a>.</p>
<p>I do not use application-level email encryption such as S/MIME or PGP. If you need a higher level of secrecy than
regular old email provides, please ask around for my signal contact or email me a file encrypted using <a class="reference external" href="https://github.com/FiloSottile/age">age</a> with one of the SSH keys listed <a class="reference external" href="https://github.com/jaseg.keys">on my github</a>. You can find both PGP and other SSH keys that I have used in the past on the
internet, but please consider these keys revoked, and do not use them to encrypt anything you send me.</p>
<div class="section" id="python-package-release-signing-key">
<h3>Python package release signing key</h3>
<p>I use this GPG key (key ID <tt class="docutils literal">ED7A208EEEC76F2D</tt>) to sign git release tags of both <a class="reference external" href="https://github.com/jaseg/gerbolyze">gerbolyze</a> and <a class="reference external" href="https://github.com/jaseg/python-mpv">python-mpv</a>:</p>
<pre class="code literal-block">
<span class="lineno"></span><span class="line">-----BEGIN PGP PUBLIC KEY BLOCK-----</span>
<span class="lineno"></span><span class="line">mDMEXom49xYJKwYBBAHaRw8BAQdA/KrWMt2MKGIZUvlQZnWjNd6i8/ZYjRsBQqEf</span>
<span class="lineno"></span><span class="line">PJ8pJ+20NHB5dGhvbi1tcHYgUmVsZWFzZSBTaWduaW5nIEtleSA8cHl0aG9uLW1w</span>
<span class="lineno"></span><span class="line">dkBqYXNlZy5kZT6IlgQTFggAPhYhBONvdTB/Cg7C0UX/XO16II7ux28tBQJeibj3</span>
<span class="lineno"></span><span class="line">AhsDBQkSzAMABQsJCAcCBhUKCQgLAgQWAgMBAh4BAheAAAoJEO16II7ux28thRYA</span>
<span class="lineno"></span><span class="line">/3Yl1RdeUGor6K0RTxce9TIBB+DpLNupJgB9f6onuocpAQC614zQ/RQ6rkGTHCwA</span>
<span class="lineno"></span><span class="line">ElFClWRQ5eppj0jpAuH15udqAbg4BF6JuPcSCisGAQQBl1UBBQEBB0A0mrXSv6rj</span>
<span class="lineno"></span><span class="line">ajCmZR4H4OtowAx477YS+yWARqo1NtdgJwMBCAeIfgQYFggAJhYhBONvdTB/Cg7C</span>
<span class="lineno"></span><span class="line">0UX/XO16II7ux28tBQJeibj3AhsMBQkSzAMAAAoJEO16II7ux28tMZwBAIUpHHvP</span>
<span class="lineno"></span><span class="line">gRW2jQuzdw1r06kItfFk/0t+mgNUQ2+vtbhzAP98BoWx7lv+bvlIbBaVgLldusj0</span>
<span class="lineno"></span><span class="line">pHnZI/0y3ksMBkdbBw==</span>
<span class="lineno"></span><span class="line">=Mr6G</span>
<span class="lineno"></span><span class="line">-----END PGP PUBLIC KEY BLOCK-----
</span></pre>
</div>
</div>
<div class="section" id="about-this-site">
<h2>About this site</h2>
<p>This site is made with the hugo static site generator. I made the theme myself, feel free to grab a copy at
<a class="reference external" href="https://git.jaseg.de/blog.git/tree/themes/conspiracy?h=main">git.jaseg.de</a>. The nifty auto-reflowing code embeds are
made with some CSS magic I made that you can find in <a class="reference external" href="https://git.jaseg.de/blog.git/tree/themes/conspiracy/assets/css/style.css?h=main&amp;id=2fd22e30ce176d8d8a641fd371ad1623b082eaaf#n367">style.css</a>.
The body text is typeset in Roboto Slab, created by <a class="reference external" href="https://christianrobertson.com/">Christian Robertson</a> while
working at Google. The headlines are set in Nyght Serif, a font by <a class="reference external" href="https://linktr.ee/mkobuzan">Maksym Kobuzan</a>.
Check out their other fonts, their work is beautiful! Source code is typeset in Fira Code, a derivate by ... from
Mozilla's <a class="reference external" href="https://github.com/mozilla/Fira">Fira Mono</a> font, designed by <a class="reference external" href="https://spiekermann.com/">Erik Spiekermann</a>, <a class="reference external" href="https://carrois.com/">Ralph du Carrois</a>, <a class="reference external" href="https://anjameiners.com/de/hallo/">Anja Meiners</a> and Botio Nikoltchev of Carrois Type Design, now succeeded by <a class="reference external" href="https://bboxtype.com/typefaces/FiraMono/#!layout=specimen">bBoxType</a> , and Patryk Adamczyk of Mozilla. The photo of mountains
that's used in the background of this site is by <a class="reference external" href="https://www.conti.photos/">Fabrizio Conti</a> and can be found on
<a class="reference external" href="https://unsplash.com/photos/TUmjK7ZJgbI">Unsplash</a>.</p>
</div>
</div>
</main><footer>
Copyright © 2025 Jan Sebastian Götte
/ <a href="/about/">About</a>
/ <a href="/imprint/">Imprint</a>
</footer>
<script type="text/javascript" src="/pagefind/pagefind-ui.js" defer></script>
<script>
window.addEventListener('DOMContentLoaded', (event) => {
new PagefindUI({element: "#search", showSubResults: true});
});
</script>
<script type="speculationrules">
{
"prerender": [
{
"source": "document",
"where": {
"and": [
{"href_matches": "/*"}
]
},
"eagerness": "moderate"
}
]
}
</script>
</body>
</html>

6
archetypes/default.md
View file

@ -1,6 +0,0 @@
---
title: "{{ replace .Name "-" " " | title }}"
date: {{ .Date }}
draft: true
---

0
content/blog/8seg/8seg-digit-circuit.png → blog/8seg/8seg-digit-circuit.png
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 92 KiB

After

Width:  |  Height:  |  Size: 92 KiB

Before After
Before After

266
content/blog/8seg/index.rst → blog/8seg/index.html
View file

@ -1,37 +1,71 @@
---
title: "8seg Technical Overview"
date: 2023-12-26T15:26:00+01:00
summary: >
8seg is a large-scale LED light art installation that displays text on a 1.5 meter high, 30 meter wide
8-segment display made from cheap LED tape.
---
<!DOCTYPE html>
<html><head>
<meta charset="utf-8">
<title>8seg Technical Overview | Home</title>
<meta name="description" content="">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="mobile-web-app-capable" content="yes">
<meta name="color-scheme" content="dark light">
<link rel="stylesheet" href="/style.css">
<link rel="preload" href="/fonts/roboto_slab/RobotoSlab-VariableFont_wght.ttf" as="font" type="font/ttf" crossorigin />
<link rel="preload" href="/fonts/nyght-serif-main/fonts/WEB/NyghtSerif-Regular.woff2" as="font" type="font/woff2" crossorigin />
<link rel="preload" href="/fonts/nyght-serif-main/fonts/WEB/NyghtSerif-Bold.woff2" as="font" type="font/woff2" crossorigin />
<link rel="preload" href="/fonts/nyght-serif-main/fonts/WEB/NyghtSerif-BoldItalic.woff2" as="font" type="font/woff2" crossorigin />
</head>
<body><nav>
<div class="internal">
<a href="/" title="Home">Home</a>
<a href="/blog/" title="Blog">Blog</a>
<a href="/projects/" title="Projects">Projects</a>
<a href="/about/" title="About">About</a>
</div>
<div class="search">
<div id="search"></div>
</div>
<div class="external">
<a href="https://git.jaseg.de/" title="cgit">cgit</a>
<a href="https://github.com/jaseg" title="Github">Github</a>
<a href="https://gitlab.com/neinseg" title="Gitlab">Gitlab</a>
<a href="https://chaos.social/@jaseg" title="Mastodon">Mastodon</a>
</span>
</nav>
Prologue
--------
<header>
<h1>8seg Technical Overview</h1>
<ul class="breadcrumbs">
<li><a href="/">jaseg.de</a></li>
<li><a href="/blog/">Blog</a></li><li><a href="/blog/8seg/">8seg Technical Overview</a></li>
</ul>
<strong>2023-12-26</strong>
</header>
<main data-pagefind-body>
<div class="document">
German hacker culture has this intense love for things that light up in colorful ways. Like for many others in this
<div class="section" id="prologue">
<h2>Prologue</h2>
<p>German hacker culture has this intense love for things that light up in colorful ways. Like for many others in this
community, I have always been fascinated by LEDs. One of the first things on my pile of unfinished projects was to build
my own LED matrix and use it to display text. When I started that project, I was still new to electronics. Back then,
commercial LED matrices were limited to red or green color only, and were very expensive, so there was an incentive to
build your own. At the same time, while individual LEDs were'nt expensive anymore, they hadn't started to be cheap yet,
either. On top of the material cost, back then there were no PCB fabs, and especially no PCB assembly houses that a
hobbyist could afford. Ultimately, I ended up never finishing this project because I felt it was more of a feat of
material wealth than of technical prowess.
Over time, LEDs came down in price, and peoople started using them in all sorts of fun things. Around the mid-2010s,
material wealth than of technical prowess.</p>
<p>Over time, LEDs came down in price, and peoople started using them in all sorts of fun things. Around the mid-2010s,
cheap-ish, ready-made tapes and chains of RGB LEDs that included WS2811 or similar digitally controllable driver chips
led to a cambrian explosion in projects involving large amounds of colorful LEDs since suddenly, all you needed was an
arduino and a beefy power supply to individually control an almost unlimited number of these LEDs.
Today, LED technology has advanced even furhter, to a point where now you can buy staggering quantities of the second
arduino and a beefy power supply to individually control an almost unlimited number of these LEDs.</p>
<p>Today, LED technology has advanced even furhter, to a point where now you can buy staggering quantities of the second
generation of these controllable LEDs that provides better color rendering embedded in all sorts of shapes, from tapes
through rings to grids. When I built the first matelight_ in 2013, the matelight's 640 individually-controllable LEDs
were *a lot*. Today, you can buy a roll with several thousand channels for about the price of a nice pizza.
The idea behind 8seg
--------------------
Living through this amazing escalation of LED technology, in 2018, I looked at a then-obsolete piece of single-color,
through rings to grids. When I built the first <a class="reference external" href="https://github.com/jaseg/matelight">matelight</a> in 2013, the matelight's 640 individually-controllable LEDs
were <em>a lot</em>. Today, you can buy a roll with several thousand channels for about the price of a nice pizza.</p>
</div>
<div class="section" id="the-idea-behind-8seg">
<h2>The idea behind 8seg</h2>
<p>Living through this amazing escalation of LED technology, in 2018, I looked at a then-obsolete piece of single-color,
dumb, non-controllable LED tape with a simple question in mind: Taking this unsophisticated artifact of yesterday's
technology, what would be the coolest thing I could build from it? Can I buld something that not only rivals, but
outmatches the modern controllable LED stuff? From that question, I set myself two goals. First, I wanted to keep the
@ -39,70 +73,58 @@ project's use of financial and labor resources reasonable. A lot of art consists
extrapolating its implementation to a ridiculous scale at the expense of the artist's time and wallet. That wasn't the
point I wanted to make. I wanted to make something cool from an obsolete technology, not prove how much patience I had
soldering. My second goal was to create something that is meaningfully controllable. Controllability is much harder with
these dumb LED tapes, but it is possible nontheless, and I wanted to test out how far you could go with it.
After thinking through a number of possibilities, I settled on the basics of the 8seg design I ended up realizing. The
these dumb LED tapes, but it is possible nontheless, and I wanted to test out how far you could go with it.</p>
<p>After thinking through a number of possibilities, I settled on the basics of the 8seg design I ended up realizing. The
installation would be a banner-style display consisting of a series of characters made from non-controllable LED tape.
The banner can be rigged up in any convenient air space, bending and folding to conform to the space's shape and size.
The key idea behind 8seg is that it makes up for it's lack of control fidelity with sheer size. If nothing else, this
non-controllable LED tape is *cheap*.
The design of a single 8seg character
-------------------------------------
Each 8seg character consists of 8 *segments* of LED tape that are inter-connected through small circuit boards, four in
non-controllable LED tape is <em>cheap</em>.</p>
</div>
<div class="section" id="the-design-of-a-single-8seg-character">
<h2>The design of a single 8seg character</h2>
<p>Each 8seg character consists of 8 <em>segments</em> of LED tape that are inter-connected through small circuit boards, four in
the corners, and one in the center. As it turns out, 8 segments arranged in this shape are enough to display all of the
English language's alphabet as well as numbers in a weird, but readable form.
The electrical design of an 8seg character has one weird trick at its core. To avoid having to run a bunch of wires from
English language's alphabet as well as numbers in a weird, but readable form.</p>
<p>The electrical design of an 8seg character has one weird trick at its core. To avoid having to run a bunch of wires from
some kind of driver circuit board to each of the eight segments, I thought, why not use the LED tape itself instead for
power and data transmission? Wires are heavy, expensive, and annoying to solder, so if I could find a way to
interconnect the LED tape so that it can all be driven from a driver circuit located at one of the character's
junctions while simultaneously powering that driver circuit, an 8seg character wouldn't need any wires at all anymore.
8seg achieves this feat using a circuit as shown in the diagram below. Interconnections between the LED tape segments
junctions while simultaneously powering that driver circuit, an 8seg character wouldn't need any wires at all anymore.</p>
<p>8seg achieves this feat using a circuit as shown in the diagram below. Interconnections between the LED tape segments
are done with a small circuit board in each of the four corners. The design is rotationally symmetric, and all four of
these boards are identicaly. The top right and bottom left corners simply use the back side of the same circuit board
used in the top left and bottom right corners.
.. image:: 8seg-digit-circuit.png
The driver circuit sits at the center of the character and directly connects to the four diagonal segments. The key
used in the top left and bottom right corners.</p>
<img alt="8seg-digit-circuit.png" src="8seg-digit-circuit.png" />
<p>The driver circuit sits at the center of the character and directly connects to the four diagonal segments. The key
thought behind 8seg's driving scheme is that there are two common phases wound through the display in a zig-zag pattern
as shown in red and blue in the schema below. These phases alternate their polarity at a high frequency. Each segment
has its negative pole connected to one of these two phases, and can be turned on by the driver while that phase is low
and the other phase is high. While a phase is high, the LEDs on all segments connected to that phase are reverse-biased,
and thus these segments remain dark.
The positive poles of all segments are connected to the driver circuit in the center through a spiral pattern. Each arm
and thus these segments remain dark.</p>
<p>The positive poles of all segments are connected to the driver circuit in the center through a spiral pattern. Each arm
of the spiral is made up of two segments, one diagonal on the inside, and one horizontal or vertical on the outside.
The two segments on each spiral arm are on different phases, one on each of the two phases. Thus, during a single cycle
of the two phases alternating polarity, first one of the two segments has its polarity the right way around, then the
other. The driver can turn on the active segment by connecting the spiral control line to the positive LED supply
voltage.
Both phases cross at the center where the driver circuit is located, so the driver can power itself from the two phases
using a simple full bridge rectifier.
Saving copper with point of load regulation
-------------------------------------------
In the beginning, I experimented with the design above, putting 12V AC on the two phases, and letting the driver switch
its derived LED supply using some cheap MOSFETs. This simple design totally works, but it has an important shortcoming.
8seg is designed to be physically *very* large. This means that not only does it have a large number of LEDs that
voltage.</p>
<p>Both phases cross at the center where the driver circuit is located, so the driver can power itself from the two phases
using a simple full bridge rectifier.</p>
</div>
<div class="section" id="saving-copper-with-point-of-load-regulation">
<h2>Saving copper with point of load regulation</h2>
<p>In the beginning, I experimented with the design above, putting 12V AC on the two phases, and letting the driver switch
its derived LED supply using some cheap MOSFETs. This simple design totally works, but it has an important shortcoming.</p>
<p>8seg is designed to be physically <em>very</em> large. This means that not only does it have a large number of LEDs that
together need a lot of current, it also has to transmit all of that current across significant physical distances. The
consequence of this was that in the initial design, I was looking at either needing hundreds of Euros worth of copper
cables, or burning hundreds of Watts of electricity into heat if I were to use thinner cables. In this case, cables act
like resistors. In a resistor, power dissipation rises with the square of the current inside the cable. This is bad for
8seg since it means halving the amount of copper in those wires increases power dissipation in these wires fourfold.
Despite that downside, this square law does come with an upside, too. If we assume we have wires of a particular fixed
8seg since it means halving the amount of copper in those wires increases power dissipation in these wires fourfold.</p>
<p>Despite that downside, this square law does come with an upside, too. If we assume we have wires of a particular fixed
diameter, if we can halve the current through those wires, we can quarter the wires' power dissipation. If we want to
deliver the same amount of power to the LEDs as before, to halve wire current, we have to double the voltage, and add
some circuitry on the drivers to convert that increased voltage back down to close to our LED tape's nominal 12V.
Alas, simply doubling the voltage leads to one question: How is it that we can pass double the voltage through our LED
some circuitry on the drivers to convert that increased voltage back down to close to our LED tape's nominal 12V.</p>
<p>Alas, simply doubling the voltage leads to one question: How is it that we can pass double the voltage through our LED
tape to the center control circuit? Isn't the LED tape made for 12V operation only, not 24V? The answer to this
apparent problem is that the center is connected to the AC bus voltage only through the negative side of the LED tapes,
and controls their positive sides to turn them on or off. The AC bus voltage never appears directly across any single of
@ -111,23 +133,20 @@ the segment control transistors with that instead of feeding them straight from
the segments with 12V. The only difference between this circuit and the straight 12V variant is that now, during OFF
times, the LED tapes see a negative 24 v across them. To make sure that's not a problem, I tested a number of them with
different LED colors and from different manufacturers, and all of them held up past the 50 V I could easily generate
with my lab power supply.
Synchronous rectification
-------------------------
I implemented the point-of-load regulation in a new revision of the center circuit, and built a prototype digit. When I
with my lab power supply.</p>
</div>
<div class="section" id="synchronous-rectification">
<h2>Synchronous rectification</h2>
<p>I implemented the point-of-load regulation in a new revision of the center circuit, and built a prototype digit. When I
tested this prototype, to my dismay, I noticed some really strange behavior. In my tests, the LED tape did not properly
light up, and when I checked the voltages with my oscilloscope, I noticed that the center circuit's ground was floating
several volts *below* the AC bus voltage's negative phase. How come?
After some head-scratching, I found that this problem was due to a simple instance of Kirchhoff's current law. Consider
several volts <em>below</em> the AC bus voltage's negative phase. How come?</p>
<p>After some head-scratching, I found that this problem was due to a simple instance of Kirchhoff's current law. Consider
the point where the AC bus voltage's currently negative phase enters the center circuit board. Let's say that we
dissipate 24 Watts in the segments' LEDs. In this case, at 24 Volts, 1 Ampère will flow into the center circuit's
terminal connected to the currently positive phase, and out from the center circuit's terminal connected to the
currently negative phase.
Now consider the current through the LED tape. During one half-cycle of the AC bus, the center circuit can only address
currently negative phase.</p>
<p>Now consider the current through the LED tape. During one half-cycle of the AC bus, the center circuit can only address
the four segments that have their negative rail connected to the currently negative phase of the AC bus. If one of these
four segments is currently on and dissipating our 24 Watts, that segment will be fed 2 Ampère of current from the center
circuit through its positive rail. My mistake was that I did not consider what happened to the return current here.
@ -136,22 +155,20 @@ circuit, and herein lies the issue: That negative rail is where our center circu
means that according to Kirchhoff's current law, the 1 A flowing out from the center circuit at its input are adding up
with the 2 A flowing into it. The result of this is that in the currently positive phase's connection, we get 1 A
flowing into the center circuit, while in the negative phase connection, we get (-1) + (+2) resulting in another 1 A
flowing into it! The only terminal where current flows *out* of the center circuit is the positive terminal connected to
the active segment, out of which 2 A of current are flowing.
The big problem with this confusing scenario is that this means the bridge recitifier in our center circuit cannot work,
flowing into it! The only terminal where current flows <em>out</em> of the center circuit is the positive terminal connected to
the active segment, out of which 2 A of current are flowing.</p>
<p>The big problem with this confusing scenario is that this means the bridge recitifier in our center circuit cannot work,
since its negative-side diodes are reverse biased while any of the segments are on. We can't just add more diodes here,
since that would just short both AC bus rails together. Instead, the solution is to add one rather chonky MOSFET in
parallel with each of the two negative-side diodes of the bridge rectifier that are controlled by the center circuit to
act as a sort of synchronous rectifier. When we turn on one of the segments, we have to turn on the MOSFET on the
currently negative rail to allow the segment's return current to bypass the bridge rectifier's negative-side diode. Fun
fact: If we turn on the wrong MOSFET out of the pair, we short the AC bus, resulting in a very quick end of life for that
poor MOSFET.
Power line data communication
-----------------------------
As we saw above, the driver providing power to a string of digits has to continuously alternate the polarity of its
poor MOSFET.</p>
</div>
<div class="section" id="power-line-data-communication">
<h2>Power line data communication</h2>
<p>As we saw above, the driver providing power to a string of digits has to continuously alternate the polarity of its
output voltage to provide one part of the digit circuits' multiplexing. Since we want to provide the control information
to the center circuits through those same two wires, we can choose between a number of viable power line communication
schemes. These schemes usually require a beefy transmitter adding a modulation at a frequency much larger than the
@ -159,45 +176,66 @@ underlying bus frequency, and a filter circuit at each receiver to filter that s
AC waveform. In our application, I saw two issues with these classical approaches. First, they require fairly complex
circuitry, especially the beefy transmitter at the driver. Second, they are susceptible to attenuation with either
changing load or over long distances, which could potentially be a problem with the high currents and long(ish) wiring
runs 8seg needs.
Because of these disadvantages, I decided on another approach entirely. Instead of modulating our control signal on top
of the AC power waveform, we modulate our control data *into* the AC power waveform. To not interfere with the display
runs 8seg needs.</p>
<p>Because of these disadvantages, I decided on another approach entirely. Instead of modulating our control signal on top
of the AC power waveform, we modulate our control data <em>into</em> the AC power waveform. To not interfere with the display
and cause outages or flicker, and to avoid having to blank the display during transmissions, we choose a modulating
technique that leaves the proportions of negative and positive half-waves undisturbed. The practical realization of this
is that instead of alternating positive and negative half-waves, we send a positive half wave for each "one" bit, and a
negative half wave for each "zero" bit, effectively creating a phase shift keyed signal with two states with an
180-degree phase shift, with the transmitted bit rate synchronized to twice the underlying carrier frequency.
The remaining question is how one can encode arbitrary binary data into a continuous stream of ones and zeros that is
is that instead of alternating positive and negative half-waves, we send a positive half wave for each &quot;one&quot; bit, and a
negative half wave for each &quot;zero&quot; bit, effectively creating a phase shift keyed signal with two states with an
180-degree phase shift, with the transmitted bit rate synchronized to twice the underlying carrier frequency.</p>
<p>The remaining question is how one can encode arbitrary binary data into a continuous stream of ones and zeros that is
precisely 50 % ones and 50 % zeros across any time span longer than a few dozen bits. There exists a near-optimal
solution to this question from ethernet over copper twisted pairs. In ethernet, the encoded and modulated signal passes
through an isolation transformer to protect the ethernet transceiver from interference or dangerous voltages coming in
through the ethernet port. For this isolation transformer to work, the modulated ethernet signal must be exactly
balanced to avoid saturating the transformer's core with a DC offset. Ethernet solves this issue by using an encoding
known as `8b/10b encoding`_. 8b/10b encoding is named like that because it specifies a way to produce a 10 bit codeword
known as <a class="reference external" href="https://en.wikipedia.org/wiki/8b/10b_encoding">8b/10b encoding</a>. 8b/10b encoding is named like that because it specifies a way to produce a 10 bit codeword
from any 8 bit input data word while guaranteeing that the resulting codewords are always precisely balanced when
looking at two or more consecutively.
Framing
-------
Since 8b/10b encoding maps a space of 256 data words to 1024 code words, there necessarily are a number of unused code
looking at two or more consecutively.</p>
</div>
<div class="section" id="framing">
<h2>Framing</h2>
<p>Since 8b/10b encoding maps a space of 256 data words to 1024 code words, there necessarily are a number of unused code
words. While for some of them, leaving them unallocated is beneficial because it improves error tolerance by decreasing
the probability of one code word turning into another undetectably when a single one of its bits is flipped, even
accounting for that it leaves some room for other uses. In 8b/10b, these leftover code words are used for synchronizing
the receiver to the transmitter, and for framing transmissions. Synchronization is necessary for the receiver to know
where a code word stards, and 8b/10b has a handful of special "comma" code words that can be uniquely identified in a
where a code word stards, and 8b/10b has a handful of special &quot;comma&quot; code words that can be uniquely identified in a
continuous stream of received ones and zeros, because no other combination of 8b/10b code words could produce the same
sequence of ones and zeros of the comma code word anywhere.
The leftover code words that are not commas are useful, too. They can be used, for instance, as filler code words
betwene actual data transmissions, or to act as framing markers denoting things like the end of a protocol message.
The 8seg driver produces its modulation waveform by translating all data to be transmitted into 8b/10b codes, padding
sequence of ones and zeros of the comma code word anywhere.</p>
<p>The leftover code words that are not commas are useful, too. They can be used, for instance, as filler code words
betwene actual data transmissions, or to act as framing markers denoting things like the end of a protocol message.</p>
<p>The 8seg driver produces its modulation waveform by translating all data to be transmitted into 8b/10b codes, padding
the result with framing markers and filler codes, and copy-pasting together the corresponding AC waveform from a small
set of pre-programmed waveform transitions.
.. _matelight: https://github.com/jaseg/matelight
.. _`8b/10b encoding`: https://en.wikipedia.org/wiki/8b/10b_encoding
set of pre-programmed waveform transitions.</p>
</div>
</div>
</main><footer>
Copyright © 2025 Jan Sebastian Götte
/ <a href="/about/">About</a>
/ <a href="/imprint/">Imprint</a>
</footer>
<script type="text/javascript" src="/pagefind/pagefind-ui.js" defer></script>
<script>
window.addEventListener('DOMContentLoaded', (event) => {
new PagefindUI({element: "#search", showSubResults: true});
});
</script>
<script type="speculationrules">
{
"prerender": [
{
"source": "document",
"where": {
"and": [
{"href_matches": "/*"}
]
},
"eagerness": "moderate"
}
]
}
</script>
</body>
</html>

259
blog/css-only-code-blocks/index.html Normal file
View file

@ -0,0 +1,259 @@
<!DOCTYPE html>
<html><head>
<meta charset="utf-8">
<title>Code listings with nice line wrapping and line numbers from plain CSS | Home</title>
<meta name="description" content="">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="mobile-web-app-capable" content="yes">
<meta name="color-scheme" content="dark light">
<link rel="stylesheet" href="/style.css">
<link rel="preload" href="/fonts/roboto_slab/RobotoSlab-VariableFont_wght.ttf" as="font" type="font/ttf" crossorigin />
<link rel="preload" href="/fonts/nyght-serif-main/fonts/WEB/NyghtSerif-Regular.woff2" as="font" type="font/woff2" crossorigin />
<link rel="preload" href="/fonts/nyght-serif-main/fonts/WEB/NyghtSerif-Bold.woff2" as="font" type="font/woff2" crossorigin />
<link rel="preload" href="/fonts/nyght-serif-main/fonts/WEB/NyghtSerif-BoldItalic.woff2" as="font" type="font/woff2" crossorigin />
</head>
<body><nav>
<div class="internal">
<a href="/" title="Home">Home</a>
<a href="/blog/" title="Blog">Blog</a>
<a href="/projects/" title="Projects">Projects</a>
<a href="/about/" title="About">About</a>
</div>
<div class="search">
<div id="search"></div>
</div>
<div class="external">
<a href="https://git.jaseg.de/" title="cgit">cgit</a>
<a href="https://github.com/jaseg" title="Github">Github</a>
<a href="https://gitlab.com/neinseg" title="Gitlab">Gitlab</a>
<a href="https://chaos.social/@jaseg" title="Mastodon">Mastodon</a>
</span>
</nav>
<header>
<h1>Code listings with nice line wrapping and line numbers from plain CSS</h1>
<ul class="breadcrumbs">
<li><a href="/">jaseg.de</a></li>
<li><a href="/blog/">Blog</a></li><li><a href="/blog/css-only-code-blocks/">Code listings with nice line wrapping and line numbers from plain CSS</a></li>
</ul>
<strong>2025-07-23</strong>
</header>
<main data-pagefind-body>
<div class="document">
<p>Code listings in web pages are often a bit of a pain to use. Often, they don't wrap on small screens. Also, copy-pasting
code from a code listing often copies the line numbers along with the code. Finally, many implementations use
heavyweight HTML and/or javascript, making them slow to render (looking at you, gitlab).</p>
<p>For this blog, I wrote an implementation that renders HTML code listings entirely without JavaScript, renders line
numbers using plain CSS such that they don't get selected with the code, and that works with the browser to wrap in a
natural way while still supporting the little line continuation arrows that are used to show that a line was soft
wrapped in text editors.</p>
<p>This blog is rendered as a static site using <a class="reference external" href="https://gohugo.io/">Hugo</a> from a pile of <a class="reference external" href="https://www.sphinx-doc.org/en/master/usage/restructuredtext/index.html">RestructuredText</a> documents. RestructuredText renders
code listings using <a class="reference external" href="https://pygments.org/">Pygments</a> by default. Pygments hard-bakes the line numbers into the generated HTML, so I am using a
<a class="reference external" href="https://en.wikipedia.org/wiki/Monkey_patch">monkey-patched</a> hook that changes the line number rendering to just a bunch of empty <tt class="docutils literal">&lt;span&gt;</tt> elements. The resulting
HTML for a code block then looks like this:</p>
<pre class="code html literal-block">
<span class="lineno"></span><span class="line"><span class="p">&lt;</span><span class="nt">pre</span> <span class="na">class</span><span class="o">=</span><span class="s">&quot;code [language] literal-block&quot;</span><span class="p">&gt;</span></span>
<span class="lineno"></span><span class="line"> <span class="p">&lt;</span><span class="nt">span</span> <span class="na">class</span><span class="o">=</span><span class="s">&quot;lineno&quot;</span><span class="p">&gt;&lt;/</span><span class="nt">span</span><span class="p">&gt;</span></span>
<span class="lineno"></span><span class="line"> <span class="p">&lt;</span><span class="nt">span</span> <span class="na">class</span><span class="o">=</span><span class="s">&quot;line&quot;</span><span class="p">&gt;</span></span>
<span class="lineno"></span><span class="line"> <span class="p">&lt;</span><span class="nt">span</span> <span class="na">class</span><span class="o">=</span><span class="s">&quot;[syntax highlight token]&quot;</span><span class="p">&gt;</span>The <span class="p">&lt;/</span><span class="nt">span</span><span class="p">&gt;&lt;</span><span class="nt">span</span> <span class="na">class</span><span class="o">=</span><span class="s">&quot;[other syntax highlight token]&quot;</span><span class="p">&gt;</span>code!<span class="p">&lt;</span><span class="nt">span</span><span class="p">&gt;</span></span>
<span class="lineno"></span><span class="line"> <span class="p">&lt;/</span><span class="nt">span</span><span class="p">&gt;</span></span>
<span class="lineno"></span><span class="line"> <span class="cm">&lt;!-- ... repeat once for each source line. --&gt;</span></span>
<span class="lineno"></span><span class="line"><span class="p">&lt;/</span><span class="nt">pre</span><span class="p">&gt;</span>
</span></pre>
<p>You can find the (rather short) source of the <tt class="docutils literal">rst2html</tt> wrapper <a class="reference external" href="#rst2html-wrapper">below</a>.</p>
<div class="section" id="the-css">
<h2>The CSS</h2>
<p>This modified HTML structure of the code listing gets accompanied by some CSS to make it flow nicely. Here is a listing
of the complete CSS controlling the listing. The only bit that isn't included here is the actual syntax styling rules
for the pygments tokens.</p>
<pre class="code css literal-block">
<span class="lineno"></span><span class="line"><span class="c">/*****************************************************/</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"></span><span class="c">/* Code block formatting / syntax highlighting rules */</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"></span><span class="c">/*****************************************************/</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"></span><span class="p">.</span><span class="nc">code</span><span class="w"> </span><span class="p">{</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"> </span><span class="k">font-family</span><span class="p">:</span><span class="w"> </span><span class="s2">&quot;Fira Code&quot;</span><span class="p">;</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"> </span><span class="k">font-size</span><span class="p">:</span><span class="w"> </span><span class="mi">13</span><span class="kt">px</span><span class="p">;</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"> </span><span class="k">text-align</span><span class="p">:</span><span class="w"> </span><span class="kc">left</span><span class="p">;</span><span class="w"> </span><span class="c">/* Override default content &quot;justify&quot; alignment */</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"> </span><span class="k">white-space</span><span class="p">:</span><span class="w"> </span><span class="kc">pre-wrap</span><span class="p">;</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"> </span><span class="k">word-wrap</span><span class="p">:</span><span class="w"> </span><span class="kc">break-word</span><span class="p">;</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"> </span><span class="k">overflow-x</span><span class="p">:</span><span class="w"> </span><span class="kc">auto</span><span class="p">;</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"> </span><span class="k">display</span><span class="p">:</span><span class="w"> </span><span class="k">grid</span><span class="p">;</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"> </span><span class="k">align-items</span><span class="p">:</span><span class="w"> </span><span class="kc">start</span><span class="p">;</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"> </span><span class="k">grid-template-columns</span><span class="p">:</span><span class="w"> </span><span class="n">min-content</span><span class="w"> </span><span class="mi">1</span><span class="n">fr</span><span class="p">;</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"></span><span class="p">}</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"></span><span class="p">.</span><span class="nc">code</span><span class="w"> </span><span class="o">&gt;</span><span class="w"> </span><span class="p">.</span><span class="nc">line</span><span class="w"> </span><span class="p">{</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"> </span><span class="k">padding-left</span><span class="p">:</span><span class="w"> </span><span class="nb">calc</span><span class="p">(</span><span class="mi">2</span><span class="kt">em</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="mi">5</span><span class="kt">px</span><span class="p">);</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"> </span><span class="k">text-indent</span><span class="p">:</span><span class="w"> </span><span class="mi">-2</span><span class="kt">em</span><span class="p">;</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"> </span><span class="k">padding-top</span><span class="p">:</span><span class="w"> </span><span class="mi">2</span><span class="kt">px</span><span class="p">;</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"> </span><span class="k">min-width</span><span class="p">:</span><span class="w"> </span><span class="mi">15</span><span class="kt">em</span><span class="p">;</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"></span><span class="p">}</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"></span><span class="c">/* Make individual syntax tokens wrap anywhere */</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"></span><span class="p">.</span><span class="nc">code</span><span class="w"> </span><span class="o">&gt;</span><span class="w"> </span><span class="p">.</span><span class="nc">line</span><span class="w"> </span><span class="o">&gt;</span><span class="w"> </span><span class="nt">span</span><span class="w"> </span><span class="p">{</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"> </span><span class="k">overflow-wrap</span><span class="p">:</span><span class="w"> </span><span class="n">anywhere</span><span class="p">;</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"> </span><span class="k">white-space</span><span class="p">:</span><span class="w"> </span><span class="kc">pre-wrap</span><span class="p">;</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"></span><span class="p">}</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"></span><span class="c">/* We render line numbers in CSS! */</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"></span><span class="p">.</span><span class="nc">code</span><span class="w"> </span><span class="o">&gt;</span><span class="w"> </span><span class="p">.</span><span class="nc">lineno</span><span class="w"> </span><span class="p">{</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"> </span><span class="k">counter-increment</span><span class="p">:</span><span class="w"> </span><span class="n">lineno</span><span class="p">;</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"> </span><span class="k">word-break</span><span class="p">:</span><span class="w"> </span><span class="n">keep-all</span><span class="p">;</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"> </span><span class="k">margin</span><span class="p">:</span><span class="w"> </span><span class="mi">0</span><span class="p">;</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"> </span><span class="k">padding-left</span><span class="p">:</span><span class="w"> </span><span class="mi">15</span><span class="kt">px</span><span class="p">;</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"> </span><span class="k">padding-right</span><span class="p">:</span><span class="w"> </span><span class="mi">5</span><span class="kt">px</span><span class="p">;</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"> </span><span class="k">overflow</span><span class="p">:</span><span class="w"> </span><span class="kc">clip</span><span class="p">;</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"> </span><span class="k">position</span><span class="p">:</span><span class="w"> </span><span class="kc">relative</span><span class="p">;</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"> </span><span class="k">text-align</span><span class="p">:</span><span class="w"> </span><span class="kc">right</span><span class="p">;</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"> </span><span class="k">color</span><span class="p">:</span><span class="w"> </span><span class="nf">var</span><span class="p">(</span><span class="nv">--c-text-muted</span><span class="p">);</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"> </span><span class="k">border-right</span><span class="p">:</span><span class="w"> </span><span class="mi">1</span><span class="kt">px</span><span class="w"> </span><span class="kc">solid</span><span class="w"> </span><span class="nf">var</span><span class="p">(</span><span class="nv">--c-fg-highlight</span><span class="p">);</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"> </span><span class="k">align-self</span><span class="p">:</span><span class="w"> </span><span class="kc">stretch</span><span class="p">;</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"></span><span class="p">}</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"></span><span class="c">/* We also handle line continuation markers in CSS. */</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"></span><span class="p">.</span><span class="nc">code</span><span class="w"> </span><span class="o">&gt;</span><span class="w"> </span><span class="p">.</span><span class="nc">lineno</span><span class="p">::</span><span class="nd">after</span><span class="w"> </span><span class="p">{</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"> </span><span class="k">position</span><span class="p">:</span><span class="w"> </span><span class="kc">absolute</span><span class="p">;</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"> </span><span class="k">right</span><span class="p">:</span><span class="w"> </span><span class="mi">5</span><span class="kt">px</span><span class="p">;</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"> </span><span class="k">content</span><span class="p">:</span><span class="w"> </span><span class="s2">&quot;\a↳\a↳\a↳\a↳\a↳\a↳\a↳\a↳\a↳\a↳\a↳\a↳\a↳\a↳\a↳\a↳\a↳\a↳&quot;</span><span class="p">;</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"> </span><span class="k">white-space</span><span class="p">:</span><span class="w"> </span><span class="kc">pre</span><span class="p">;</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"> </span><span class="k">color</span><span class="p">:</span><span class="w"> </span><span class="nf">var</span><span class="p">(</span><span class="nv">--c-text-muted</span><span class="p">);</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"></span><span class="p">}</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"></span><span class="c">/* Insert the actual line number */</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"></span><span class="p">.</span><span class="nc">code</span><span class="w"> </span><span class="o">&gt;</span><span class="w"> </span><span class="p">.</span><span class="nc">lineno</span><span class="p">::</span><span class="nd">before</span><span class="w"> </span><span class="p">{</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"> </span><span class="k">content</span><span class="p">:</span><span class="w"> </span><span class="nb">counter</span><span class="p">(</span><span class="n">lineno</span><span class="p">);</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"></span><span class="p">}</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"></span><span class="p">.</span><span class="nc">code</span><span class="p">::</span><span class="nd">before</span><span class="w"> </span><span class="p">{</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"> </span><span class="k">counter-reset</span><span class="p">:</span><span class="w"> </span><span class="n">lineno</span><span class="p">;</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"></span><span class="p">}</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"></span><span class="p">.</span><span class="nc">code</span><span class="w"> </span><span class="p">.</span><span class="nc">hll</span><span class="w"> </span><span class="p">{}</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"></span><span class="c">/* Following are about 50 lines that define the styling of each kind of pygments syntax highlight token. These lines</span></span>
<span class="lineno"></span><span class="line"><span class="c"> all look like the following: */</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"></span><span class="p">.</span><span class="nc">code</span><span class="w"> </span><span class="p">.</span><span class="nc">c</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="k">color</span><span class="p">:</span><span class="w"> </span><span class="nf">var</span><span class="p">(</span><span class="nv">--c-text</span><span class="p">);</span><span class="w"> </span><span class="k">font-weight</span><span class="p">:</span><span class="w"> </span><span class="mi">400</span><span class="w"> </span><span class="p">}</span><span class="w"> </span><span class="c">/* Comment */</span>
</span></pre>
<p>This CSS does a few things:</p>
<blockquote>
<ol class="arabic simple">
<li>It renders the <tt class="docutils literal">&lt;pre&gt;</tt> code listing element using a two-column CSS <tt class="docutils literal">display: grid</tt> layout. The left column is
used for the line numbers, and the right column is used for the code lines.</li>
<li>It numbers the lines using a <a class="reference external" href="https://developer.mozilla.org/en-US/docs/Web/CSS/counter">CSS Counter</a>. CSS counters are meant for things like numbering headings and such, but
they are a perfect fit for our purpose.</li>
<li>It inserts the counter value as the line number into the <tt class="docutils literal">&lt;span <span class="pre">class=&quot;lineno&quot;&gt;</span></tt> element's <tt class="docutils literal">::before</tt>
pseudo-element. A side effect of using the <tt class="docutils literal">::before</tt> pseudo-element is that without doing anything extra, the
line numbers will remain outside of the normal text selection so they will neither be highlighted when selecting
listing content, nor will they be copied when copy/pasting the listing content.</li>
<li>It inserts a string of <tt class="docutils literal"><span class="pre">&quot;\a↳\a↳\a↳\a↳\a↳\a↳\a↳\a↳\a↳\a↳\a↳\a↳\a↳\a↳\a↳\a↳\a↳\a↳&quot;</span></tt> into the line number span's
<tt class="docutils literal">::after</tt> pseudo-element. This string evaluates to a sequence of unicode arrows separated by line breaks, and
starting with an empty line. The <tt class="docutils literal">::after</tt> pseudo-element is positioned using <tt class="docutils literal">position: absolute</tt>, and the
parent <tt class="docutils literal">&lt;span <span class="pre">class=&quot;lineno&quot;&gt;</span></tt> has <tt class="docutils literal">position: relative</tt> set. This way, the arrow pseudo-element gets placed on
top of the lineno span without affecting the layout at all. By setting <tt class="docutils literal">overflow: clip</tt> on the parent <tt class="docutils literal">&lt;span
<span class="pre">class=&quot;lineno&quot;&gt;</span></tt>, the arrow pseudo-element gets cut off vertically wherever the parent lineno element naturally
ends.</li>
</ol>
</blockquote>
<p>The line number span is inserted into the parent <tt class="docutils literal">&lt;pre&gt;</tt> element's CSS grid using <tt class="docutils literal"><span class="pre">align-self:</span> stretch</tt>, which
causes it to vertically stretch to fill the available space. Since the line number span only contains the line number,
its minimum height is a single line. As a result, it will stretch higher only when the corresponding code line in the
right grid column stretches vertically because of line wrapping. When that happens, part of the arrow pseudo-element
starts showing through from behind the <tt class="docutils literal">overflow: clip</tt> of the line number span, and one arrow gets rendered for each
wrapped listing line.</p>
<p>When the page is too narrow, we don't want the code listing's lines to wrapp into a column of single characters. To
prevent that, we simply set a <tt class="docutils literal"><span class="pre">min-width</span></tt> on the <tt class="docutils literal">&lt;span <span class="pre">class=&quot;line&quot;&gt;</span></tt> in the right column, and set <tt class="docutils literal"><span class="pre">overflow-x:</span>
auto</tt> on the listing <tt class="docutils literal">&lt;pre&gt;</tt>. This results in a horizontal scroll bar appearing whenever the listing gets too narrow.</p>
<p>You can try out the line wrapping by resizing this page!</p>
</div>
<div class="section" id="rst2html-wrapper">
<h2>rst2html wrapper</h2>
<p>Here is the python <tt class="docutils literal">rst2html</tt> wrapper that monkey-patches code rendering. I made hugo invoke this while building the
page by simply overriding the <tt class="docutils literal">PATH</tt> environment variable.</p>
<pre class="code python literal-block">
<span class="lineno"></span><span class="line"><span class="ch">#!/usr/bin/env python3</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"></span><span class="c1"># Based on https://gist.github.com/mastbaum/2655700 for the basic plugin scaffolding</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"></span><span class="kn">import</span><span class="w"> </span><span class="nn">sys</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"></span><span class="kn">import</span><span class="w"> </span><span class="nn">re</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"></span><span class="kn">import</span><span class="w"> </span><span class="nn">docutils.core</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"></span><span class="kn">from</span><span class="w"> </span><span class="nn">docutils.transforms</span><span class="w"> </span><span class="kn">import</span> <span class="n">Transform</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"></span><span class="kn">from</span><span class="w"> </span><span class="nn">docutils.nodes</span><span class="w"> </span><span class="kn">import</span> <span class="n">TextElement</span><span class="p">,</span> <span class="n">Inline</span><span class="p">,</span> <span class="n">Text</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"></span><span class="kn">from</span><span class="w"> </span><span class="nn">docutils.parsers.rst</span><span class="w"> </span><span class="kn">import</span> <span class="n">Directive</span><span class="p">,</span> <span class="n">directives</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"></span><span class="kn">from</span><span class="w"> </span><span class="nn">docutils.writers.html4css1</span><span class="w"> </span><span class="kn">import</span> <span class="n">Writer</span><span class="p">,</span> <span class="n">HTMLTranslator</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"></span><span class="k">class</span><span class="w"> </span><span class="nc">UnfuckedHTMLTranslator</span><span class="p">(</span><span class="n">HTMLTranslator</span><span class="p">):</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"></span> <span class="k">def</span><span class="w"> </span><span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"></span> <span class="nb">super</span><span class="p">()</span><span class="o">.</span><span class="fm">__init__</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">)</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"></span> <span class="bp">self</span><span class="o">.</span><span class="n">in_literal_block</span> <span class="o">=</span> <span class="kc">False</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"></span> <span class="k">def</span><span class="w"> </span><span class="nf">visit_literal_block</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">node</span><span class="p">):</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"></span> <span class="c1"># Insert an empty &quot;lineno&quot; span before each line. We insert the line numbers using pure CSS in a ::before</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"></span> <span class="c1"># pseudo-element. This has the added advantage that the line numbers don't get included in text selection.</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"></span> <span class="c1"># These line number spans are also used to show line continuation markers when a line is wrapped.</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"></span> <span class="bp">self</span><span class="o">.</span><span class="n">in_literal_block</span> <span class="o">=</span> <span class="kc">True</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"></span> <span class="bp">self</span><span class="o">.</span><span class="n">body</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">starttag</span><span class="p">(</span><span class="n">node</span><span class="p">,</span> <span class="s1">'pre'</span><span class="p">,</span> <span class="n">CLASS</span><span class="o">=</span><span class="s1">'literal-block'</span><span class="p">))</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"></span> <span class="bp">self</span><span class="o">.</span><span class="n">body</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="s1">'&lt;span class=&quot;lineno&quot;&gt;&lt;/span&gt;&lt;span class=&quot;line&quot;&gt;'</span><span class="p">)</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"></span> <span class="k">def</span><span class="w"> </span><span class="nf">depart_literal_block</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">node</span><span class="p">):</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"></span> <span class="bp">self</span><span class="o">.</span><span class="n">in_literal_block</span> <span class="o">=</span> <span class="kc">False</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"></span> <span class="bp">self</span><span class="o">.</span><span class="n">body</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="s1">'</span><span class="se">\n</span><span class="s1">&lt;/span&gt;&lt;/pre&gt;</span><span class="se">\n</span><span class="s1">'</span><span class="p">)</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"></span> <span class="k">def</span><span class="w"> </span><span class="nf">visit_Text</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">node</span><span class="p">):</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"></span> <span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">in_literal_block</span><span class="p">:</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"></span> <span class="k">for</span> <span class="n">match</span> <span class="ow">in</span> <span class="n">re</span><span class="o">.</span><span class="n">finditer</span><span class="p">(</span><span class="s1">'([^</span><span class="se">\n</span><span class="s1">]*)(</span><span class="se">\n</span><span class="s1">|$)'</span><span class="p">,</span> <span class="n">node</span><span class="o">.</span><span class="n">astext</span><span class="p">()):</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"></span> <span class="n">text</span><span class="p">,</span> <span class="n">end</span> <span class="o">=</span> <span class="n">match</span><span class="o">.</span><span class="n">groups</span><span class="p">()</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"></span> <span class="k">if</span> <span class="n">text</span><span class="p">:</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"></span> <span class="nb">super</span><span class="p">()</span><span class="o">.</span><span class="n">visit_Text</span><span class="p">(</span><span class="n">Text</span><span class="p">(</span><span class="n">text</span><span class="p">))</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"></span> <span class="k">if</span> <span class="n">end</span> <span class="o">==</span> <span class="s1">'</span><span class="se">\n</span><span class="s1">'</span><span class="p">:</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"></span> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">parent</span><span class="p">,</span> <span class="n">Inline</span><span class="p">):</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"></span> <span class="bp">self</span><span class="o">.</span><span class="n">depart_inline</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">parent</span><span class="p">)</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"></span> <span class="bp">self</span><span class="o">.</span><span class="n">body</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="sa">f</span><span class="s1">'&lt;/span&gt;</span><span class="se">\n</span><span class="s1">&lt;span class=&quot;lineno&quot;&gt;&lt;/span&gt;&lt;span class=&quot;line&quot;&gt;'</span><span class="p">)</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"></span> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">parent</span><span class="p">,</span> <span class="n">Inline</span><span class="p">):</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"></span> <span class="bp">self</span><span class="o">.</span><span class="n">visit_inline</span><span class="p">(</span><span class="n">node</span><span class="o">.</span><span class="n">parent</span><span class="p">)</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"></span> <span class="k">else</span><span class="p">:</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"></span> <span class="nb">super</span><span class="p">()</span><span class="o">.</span><span class="n">visit_Text</span><span class="p">(</span><span class="n">node</span><span class="p">)</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"></span><span class="n">html_writer</span> <span class="o">=</span> <span class="n">Writer</span><span class="p">()</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"></span><span class="n">html_writer</span><span class="o">.</span><span class="n">translator_class</span> <span class="o">=</span> <span class="n">UnfuckedHTMLTranslator</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"></span><span class="n">docutils</span><span class="o">.</span><span class="n">core</span><span class="o">.</span><span class="n">publish_cmdline</span><span class="p">(</span><span class="n">writer</span><span class="o">=</span><span class="n">html_writer</span><span class="p">)</span>
</span></pre>
</div>
</div>
</main><footer>
Copyright © 2025 Jan Sebastian Götte
/ <a href="/about/">About</a>
/ <a href="/imprint/">Imprint</a>
</footer>
<script type="text/javascript" src="/pagefind/pagefind-ui.js" defer></script>
<script>
window.addEventListener('DOMContentLoaded', (event) => {
new PagefindUI({element: "#search", showSubResults: true});
});
</script>
<script type="speculationrules">
{
"prerender": [
{
"source": "document",
"where": {
"and": [
{"href_matches": "/*"}
]
},
"eagerness": "moderate"
}
]
}
</script>
</body>
</html>

258
blog/hsm-basics/index.html Normal file
View file

@ -0,0 +1,258 @@
<!DOCTYPE html>
<html><head>
<meta charset="utf-8">
<title>Hardware Security Module Basics | Home</title>
<meta name="description" content="">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="mobile-web-app-capable" content="yes">
<meta name="color-scheme" content="dark light">
<link rel="stylesheet" href="/style.css">
<link rel="preload" href="/fonts/roboto_slab/RobotoSlab-VariableFont_wght.ttf" as="font" type="font/ttf" crossorigin />
<link rel="preload" href="/fonts/nyght-serif-main/fonts/WEB/NyghtSerif-Regular.woff2" as="font" type="font/woff2" crossorigin />
<link rel="preload" href="/fonts/nyght-serif-main/fonts/WEB/NyghtSerif-Bold.woff2" as="font" type="font/woff2" crossorigin />
<link rel="preload" href="/fonts/nyght-serif-main/fonts/WEB/NyghtSerif-BoldItalic.woff2" as="font" type="font/woff2" crossorigin />
</head>
<body><nav>
<div class="internal">
<a href="/" title="Home">Home</a>
<a href="/blog/" title="Blog">Blog</a>
<a href="/projects/" title="Projects">Projects</a>
<a href="/about/" title="About">About</a>
</div>
<div class="search">
<div id="search"></div>
</div>
<div class="external">
<a href="https://git.jaseg.de/" title="cgit">cgit</a>
<a href="https://github.com/jaseg" title="Github">Github</a>
<a href="https://gitlab.com/neinseg" title="Gitlab">Gitlab</a>
<a href="https://chaos.social/@jaseg" title="Mastodon">Mastodon</a>
</span>
</nav>
<header>
<h1>Hardware Security Module Basics</h1>
<ul class="breadcrumbs">
<li><a href="/">jaseg.de</a></li>
<li><a href="/blog/">Blog</a></li><li><a href="/blog/hsm-basics/">Hardware Security Module Basics</a></li>
</ul>
<strong>2019-05-17</strong>
</header>
<main data-pagefind-body>
<div class="document">
<p>On May 17 2019 I gave a short presentation on the fundamentals of hardware security modules at the weekly seminar of
Prof. Mori's security research working group at Waseda University. The motivation for this was that outside of low-level
hardware security people and people working in the financial industry HSMs are not thought about that often. In
particular most network or systems security people would not consider them an option. Also it could turn out to be
really interesting to think about what could be done with an HSM in conjunction with modern cryptography (instead of
just plain old RSA-OAEP and AES-CBC).</p>
<p><a class="reference external" href="mori_semi_hsm_talk_web.pdf">Click here to download a PDF with the slides for this talk.</a></p>
<div class="section" id="ideas-for-research-in-hsms">
<h2>Ideas for research in HSMs</h2>
<p>Preparing for this talk brought me back to some research ideas I've been working on for a while now. Since I'm not sure
I'll find the time to properly research this topic, I thought it would be great to write down some rought outlines first
for future reference.</p>
<div class="section" id="the-problem-with-current-hsm-tech">
<h3>The Problem with current HSM tech</h3>
<p>Currently, HSMs are only used in certain specific niche applications such as certificate authority key management and
financial transaction data handling. One key reason for this is that HSMs currently don't provide the affordances that
would be needed for them to be adopted more widely by the cryptographic and security engineering community. As far as I
can tell, the two core missing affordances are:</p>
<ol class="arabic simple">
<li>To be more widely adopted, HSMs must become less expensive. Currently, they go for several tens of thousands of Euro,
which puts them outside most budgets.</li>
<li>To be more widely adopted, HSMs must provide the standardized programming interfaces familiar to cryptographic
developers. Currently, every HSM vendor has their own custom cryptographic API and a developer will have to train on
one specific vendor's tooling. Furthermore, any documentation of these internals is kept secret behind NDAs. This
constitutes a high barrier to entry, decreasing adoption in particular with young developers accustomed to
open-source ecosystems.</li>
</ol>
</div>
<div class="section" id="attacking-cost-of-implementation">
<h3>Attacking cost of implementation</h3>
<p>The first issue can be addressed by simply creating a viable low-cost alternative. There is no fundamental technical
reason for the high cost of HSMs. This cost is instead due to manufacturers trying to recoup their expenses for R&amp;D as
well as certification from the small volumes HSMs are sold in.</p>
<p>Compared to system integration and certification the pure R&amp;D cost of HSM defense mechanisms themselves is not too high
in an academic context it should be feasible to develop a sort of HSM blueprint that can then be cheaply produced by
anyone in need. Since the application areas outlined here are far from the core business areas of the clients of
established HSM vendors this would most likely not be a realistic threat to any established vendor's business and a
co-existence of both should not pose any problems in the short term.</p>
</div>
<div class="section" id="benefits-of-an-academic-hsm-standard">
<h3>Benefits of an academic HSM standard</h3>
<p>Tackling the high cost of current HSM hardware with an open-source HSM blueprint would yield
several academic advantages beyond cost reduction.</p>
<ol class="arabic simple">
<li>An open-source blueprint could serve as an academic reference design to evaluate and compare other HSM designs
against. For instance this would not only allow quantifying the effectiveness of academic security measures but also
allow an evaluation of commercial HSMs.</li>
<li>An open-source blueprint could stimulate academic research in this academically very quiet albeit commercially
important area. This research would ultimately benefit everyone employing HSMs by raising security standards in the
field. Since HSMs are never solely relied upon for overal system security both defensive and offensive security
research would yield these benefits.</li>
<li>An open-source blueprint would encourage new people to get into the field and both apply HSMs to practical problems
as well as improve HSMs themselves. Currently, this is highly discouraged due to the strictly proprietary nature of
all available systems.</li>
<li>Finally, developing an open-source HSM blueprint might yield new findings in adjacent academic areas due to the
hightly multi-disciplinary nature of security research in general and HSM design in particular.</li>
</ol>
</div>
<div class="section" id="scope-of-an-academic-hsm-standard">
<h3>Scope of an academic HSM standard</h3>
<p>An academic HSM blueprint would need to be flexible so that researchers can adapt it to their particular problem. A
modular architecture would lend itself to this flexibility. Fundamentally, there would be three components to this
architecture. First, a <strong>base</strong> containing infrastructure such as the surveillance microcontroller, power supplies,
power supply filtering and hardware DPA countermeasures, and possibly a standardized mechanical and electrical
interface.</p>
<p>Next to the base, a system integrator would put their <em>payload</em>. The nature of this payload is intentionally kept
unspecified, and it might be anything from a cryptographic microcontroller to a small embedded system such as a
raspberry pi single board computer. Keeping the <em>payload</em> open like this achieves two benefits: It gives the HSM
blueprint's user <em>their</em> familiar tooling and the hardware <em>they</em> need, allowing fast adoption. Someone well-versed in
e.g. Javascript could literally implement their cryptography in Javascript, run it on an off-the-shelf raspberry pi and
just apply the HSM blueprint around it. In addition, keeping the <em>payload</em> open reduces the scope of what needs to be
implemented. Building a general SDK on top of something like a bare ARM SoC such as a TI OMAP or a Freescale/NXP IMX
would be a considerable additional burden, on top of the actual HSM design. Keeping the <em>payload</em> open allows research
to concentrate on the actual point, the HSM design.</p>
<p>The final and most important component would be a set of <em>security measures</em> that can be combined with the base to
form the final HSM. Each of these <em>security measures</em> would entail a detailed specification of its design, manufacture
and security properties. These <em>security measures</em> could be simple things like tamper switches or potting, but could
also be complex things like security meshes.</p>
<p>Given these three components -- <em>base</em>, <em>payload</em> and <em>security measures</em> as detailed specifications any engineer should
be able to design and manufacture a HSM customized to their needs. Unifying these three components within the HSM
blueprint would be a set of reference designs. Each reference design would implement a particular parametrization of the
three architectural components with a physical hole cut out where the payload would go.. These reference designs would
for one serve to guide any implementer on the customization and integration of their own derivation from the blueprint.
In addition it would serve as an extremely simple, low-cost point of entry into the ecosystem. A curious researcher
could simply replicate the reference design and put their existing payload inside. Practically this might mean e.g. a
researcher having PCBs produced according to the design files for a reference design for a mesh-based HSM, producing
their own mesh, physically glueing a raspberry pi SBC into the middle of it, and potting the resulting system. Given the
ease of prototype PCB fabrication today this would realistically allow evaluation of HSM technologies on a budget that
is orders of magnitude less than the cost of current HSMs.</p>
</div>
</div>
<div class="section" id="research-ideas-for-tamper-detection-mechanisms">
<h2>Research ideas for tamper detection mechanisms</h2>
<p>The core component of an HSM blueprint would be a suite of tamper detection mechanisms. Following are a few ideas on how
to improve on the current state of the art of membrane tamper switches plus temperature sensors plus PCB and printed
security meshes plus potting.</p>
<div class="section" id="diy-or-small-lab-mesh-production">
<h3>DIY or small lab mesh production</h3>
<p><strong>Analog sensing</strong> meshes are a proven technology where instead of just monitoring for continuity and shorts, analog
parameters of the mesh traces such as inductance and mutual capacitance are monitored. In 2019, <a class="reference external" href="https://tches.iacr.org/index.php/TCHES/article/view/7334">Immler et al. published
a paper</a> where took this principle and turned it all the
way up. They directly derived a cryptographic secret from the analog properties of their HSM's security mesh in an
attempt to built a <a class="reference external" href="https://en.wikipedia.org/wiki/Physical_unclonable_function">Physically Unclonable Function, or PUF</a>. The idea with PUFs is that they reproduce some entropy
that comes from random tolerances of their production process. The same PUF will always yield (approximately) the same
key, but since you cannot control these random production variations, in practice the resulting PUF cannot be cloned.
Note however, that its secrets can of course be copied if you find a way to read them out.</p>
<p>As Immler et al. demonstrated in their paper, you don't need any secret sauce to create an analog mesh sensing circuit.
All you need are a bunch of (admittedly, expensive) off-the-shelf analog ICs. The interesting bit here is that by
applying more advanced analog sensing, weaknesses of an otherwise coarse mesh desing could maybe be alleviated. That is,
instead of monitoring a very fine mesh for continuity, you could instead closely monitor inductance and capacitance of a
more coarse mesh. This trade-off between sensing circuit complexity (resp. cost) and mesh production capabilities may
allow someone with a poorly equipped lab to still make a decent HSM. The question is, how do you produce a &quot;decent&quot; mesh
given only basic tools? Here are some ideas.</p>
<p><strong>3D metal patterning techniques</strong> refers to any technique for producing thin, patterned metal structures on a
three-dimensional plastic substrate. The basic process would consist of 3D-printing the polymer substrate, depositing a
thin metal layer on top and then patterning this metal layer. A good starting point here would be the recent work of
<a class="reference external" href="https://www.youtube.com/watch?v=Z228xymQYho">Ben Kraznow</a> on this exact thing.</p>
<p><strong>Copper filament methods</strong> would be any method embedding copper wire from a spool in some resin or other matrix. This
could mean either of a systematic approach of carefully winding or folding the copper wire into patterns or a
non-systematic approach of simply stuffing a large tangle of copper wire into a small space. The main challenge with the
former would be to find a non-tedious way of production. The main challenge with the latter would be to find process
parameters that guarantee complete coverage of the HSM without holes or other areas of lower sensitivity to intrusions.
Both approaches would require careful consideration of the overall design including the polymer resin supporting
structure to ensure sensitivity against attacks since copper wire is mechanically much stronger than the micrometre-thin
metal coatings used in patterning techniques.</p>
</div>
<div class="section" id="envelope-measurement">
<h3>Envelope measurement</h3>
<p>Finally, I think there is another set of currently under-utilized tamper-detection methods that would be very
interesting to explore. I am not aware of an academic term for these, so I am just going to dub them <em>envelope
measurement</em> here.</p>
<p>The fundamental apporach of a mesh is to build a physical security envelope (the mesh) that physically detects when it
is disturbed (open or short circuits). This approach works well but has the disadvantage that these meshes are rather
complex to manufacture since effectively every part of them is acting as a sensing element. A conceptually more complex
but in practice potentially simpler approach might be to split the functions of security envelope and sensing element.
This would mean that in place of the mesh, some form of passive element such as metal foil forms the security envelope
which is then checked for tampering using a very sensitive sensor inside. This remote-sensing approach might simplify
the manufacture of the envelope itself and thus yield a design that is more easily customized. Following are a few ideas
on how to approach this envelope measurement problem.</p>
<p><strong>Ultrasonic</strong> If the HSM is potted, a few ultrasonic transducers could be added inside the potting. With several
transducers, any one could be used to transmit ultrasound while the others measure complex phase and energy of the
signal they receive. The circuitry for this could be made fairly simple if using a static transmit frequency or a low
chirp rate by using a homodyne receiver built around a comparator fed into some timers. This approach would likely
detect any mechanical attack and would also rule out chemical attacks involving liquids (though starting from which
amount of liquid depends on receiver sensitivity). The main disadvantages might be high power consumption and cost and
size of the ultrasonic transducers. Traditional cheap transducers made for air as a transmission medium are fairly large
and might not adequately couple into potting compound. If somehow one could convince a standard small piezo element to
do the same job that would be great as far as cost and size are concerned. A concern in some fringe use cases might be
suceptibility to ambient noise, though this could easily be reduced at the expense of space and heat dissipation
capacity by adding sound dampening on the outside. A likely attack vector against this approach might be using a laser
cutter to drill a hole through the potting compound, then inserting probes carefully chosen to not couple too much
to the potting compound ultrasonically.</p>
<p><strong>Light</strong> In either an unpotted HSM or one potted with a transparent (at some wavelengths) potting compound one could
embed LEDs and photodiodes in a similar setup to the ultrasonic setup described above. In contrast to the ultrasound,
the LEDs would literally have to light up the HSM's interior and shadows might be an issue since the HSM is likely some
flat rectangular shape. A possible solution to this would be to coat both the embedded payload and the lid with some
highly reflective paint such as some glossy silver paint or simple white paint. The basic approach might be as simple as
simply turning on several LEDs distributed throughout the HSM in turn and measuring amplitude at several photodetectors,
or as complex as doing a LIDAR-like phase measurement sweeping through a range of frequencies to determine not only
absorption but also phase/distance characteristics between emitter LED and detector photodiode. Using some high-gain TIA
along with a homodyne detector (lock-in amplifier) and changing emitter intensity, very precise measurements of both
absorption and phase might be possible, as might be measurements through almost opaque, diffuse potting compounds such
as a grey epoxide resin. The main disadvantages of this method would likely be the need to thoroughly light-proof the
entire HSM (likely by wrapping it in metal foil) and the potentially high cost of transmitter and receiver circuitry
(nice TIAs aren't cheap). To be effective against attacks using e.g. very fine drills and probes the system would likely
have to be very sensitive.</p>
<p><strong>Radar</strong> Finally, one could turn to standard radar techniques to fingerprint the inside of the HSM. The goal here would
be fingerprinting instead of mapping since only changes need to be detected. In this approach one could use homodyne
detection to improve sensitivity and reduce receiver complexity, and sweep frequencies similar to an FMCW radar (but
probably without exploiting the self-demodulation effect). Besides high cost, this approach has two disadvantages.
First, such a system would likely not go beyond 24GHz or maybe 40GHz due to component availability issues. Even at 40GHz
the wavelength in the potting compound would be in the order of magnitude of several millimeters. Fine intrusions using
some tool chosen to not interact too much with the EM field inside the HSM such as a heated ceramic needle or simply a
laser cutter might not be detectable using this approach. In any case, this system would certainly not be able to detect
small holes piercing the HSM enclosure. The HSM enclosure would have to be made into an RF shield, likely by using some
metal foil in it.</p>
<p>Overall in the author's opinion these three techniques are most promising in order <em>Light</em>, <em>Ultrasonic</em>, <em>Radar</em>. Light
would prbably provide the best sensitivity at expense of some cost. Ultrasonic might be used in conjunction with light
to cover some additional angles since it is potentially very low-cost. Radar seems hard to engineer into a solution that
works reliably and also would likely be at least an order of magnitude more expensive than the other two technologies
while not providing better sensitivity.</p>
</div>
</div>
</div>
</main><footer>
Copyright © 2025 Jan Sebastian Götte
/ <a href="/about/">About</a>
/ <a href="/imprint/">Imprint</a>
</footer>
<script type="text/javascript" src="/pagefind/pagefind-ui.js" defer></script>
<script>
window.addEventListener('DOMContentLoaded', (event) => {
new PagefindUI({element: "#search", showSubResults: true});
});
</script>
<script type="speculationrules">
{
"prerender": [
{
"source": "document",
"where": {
"and": [
{"href_matches": "/*"}
]
},
"eagerness": "moderate"
}
]
}
</script>
</body>
</html>

0
content/blog/hsm-basics/mori_semi_hsm_talk_web.pdf → blog/hsm-basics/mori_semi_hsm_talk_web.pdf
View file

102
blog/ihsm-worlds-first-diy-hsm/index.html Normal file
View file

@ -0,0 +1,102 @@
<!DOCTYPE html>
<html><head>
<meta charset="utf-8">
<title>New Paper on Inertial Hardware Security Modules | Home</title>
<meta name="description" content="">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="mobile-web-app-capable" content="yes">
<meta name="color-scheme" content="dark light">
<link rel="stylesheet" href="/style.css">
<link rel="preload" href="/fonts/roboto_slab/RobotoSlab-VariableFont_wght.ttf" as="font" type="font/ttf" crossorigin />
<link rel="preload" href="/fonts/nyght-serif-main/fonts/WEB/NyghtSerif-Regular.woff2" as="font" type="font/woff2" crossorigin />
<link rel="preload" href="/fonts/nyght-serif-main/fonts/WEB/NyghtSerif-Bold.woff2" as="font" type="font/woff2" crossorigin />
<link rel="preload" href="/fonts/nyght-serif-main/fonts/WEB/NyghtSerif-BoldItalic.woff2" as="font" type="font/woff2" crossorigin />
</head>
<body><nav>
<div class="internal">
<a href="/" title="Home">Home</a>
<a href="/blog/" title="Blog">Blog</a>
<a href="/projects/" title="Projects">Projects</a>
<a href="/about/" title="About">About</a>
</div>
<div class="search">
<div id="search"></div>
</div>
<div class="external">
<a href="https://git.jaseg.de/" title="cgit">cgit</a>
<a href="https://github.com/jaseg" title="Github">Github</a>
<a href="https://gitlab.com/neinseg" title="Gitlab">Gitlab</a>
<a href="https://chaos.social/@jaseg" title="Mastodon">Mastodon</a>
</span>
</nav>
<header>
<h1>New Paper on Inertial Hardware Security Modules</h1>
<ul class="breadcrumbs">
<li><a href="/">jaseg.de</a></li>
<li><a href="/blog/">Blog</a></li><li><a href="/blog/ihsm-worlds-first-diy-hsm/">New Paper on Inertial Hardware Security Modules</a></li>
</ul>
<strong>2021-11-23</strong>
</header>
<main data-pagefind-body>
<div class="document" id="world-s-first-diy-hsm">
<h1 class="title">World's First DIY HSM</h1>
<p>Last week, Prof. Dr. Björn Scheuermann and I have <a class="reference external" href="https://tches.iacr.org/index.php/TCHES/article/view/9290">published our first joint paper on Hardware Security Modules</a>. In our paper, we introduce Inertial Hardware Security
Modules (IHSMs), a new way of building high-security HSMs from basic components. I think the technology we demonstrate
in our paper might allow some neat applications where some civil organization deploys a service that no one, not even
they themselves, can snoop on. Anyone can built an IHSM without needing any fancy equipment, which makes me optimistic
that maybe the ideas of the <a class="reference external" href="https://www.activism.net/cypherpunk/manifesto.html">Cypherpunk movement</a> aren't obsolete
after all, despite even the word &quot;crypto&quot; having been co-opted by radical capitalist environmental destructionists.</p>
<p>An IHSM is basically an ultra-secure enclosure for something like a server or a raspberry pi that even someone with
unlimited resources would have a really hard time cracking without destroying all data stored in it. The principle of an
IHSM is the same as that of a <a class="reference external" href="http://jaseg.de/blog/hsm-basics/">normal HSM</a>. You have a payload that contains really secret data. There's really no way
to prevent an attacker with physical access to the thing from opening it given enough time and abrasive discs for their
angle grinder. So what you do instead is that you make it self-destruct its secrets within microseconds of anyone
tampering with it. Usually, such HSMs are used for storing credit card pins and other financial data. They're expensive
as fuck, all the while being about the same processing speed as a smartphone. Traditional HSMs use printed or
lithographically patterned conductive foils for their security mesh. These foils are not an off-the-shelf component and
are made in a completely custom manufacturing process. To create your own, you would have to re-engineer that entire
process and probably spend some serious money on production machines.</p>
<p>Inertial HSMs take the concept of traditional HSMs, but replace the usual tamper detection mesh with a few security mesh
PCBs. These PCBs are coarser than traditional meshes by orders of magnitude, and would alone not even be close to enough
to keep out even a moderately motivated attacker. IHSMs fix this issue by spinning the entire tamper detection mesh at
very high speed. To tamper with the mesh, an attacker would have to stop it. This, in turn, can be easily detected by
the mesh's alarm circuitry using a simple accelerometer as a rotation sensor.</p>
<p>In our paper, we have shown a working prototype of the core concepts one needs to build such an IHSM. To build an IHSM
you only need a basic electronics lab. I built the prototype in our paper at home during one of Germany's COVID
lockdowns. You can have a look at our code and CAD on <a class="reference external" href="https://git.jaseg.de/ihsm.git">my git</a>. What is missing right
now is an integration of all of these fragments into something cohesive that an interested person with the right tools
could go out and build. We are planning to release this sort of documentation at some point, but right now we are
focusing our effort on the next iteration of the design instead. Stay tuned for updates ;)</p>
</div>
</main><footer>
Copyright © 2025 Jan Sebastian Götte
/ <a href="/about/">About</a>
/ <a href="/imprint/">Imprint</a>
</footer>
<script type="text/javascript" src="/pagefind/pagefind-ui.js" defer></script>
<script>
window.addEventListener('DOMContentLoaded', (event) => {
new PagefindUI({element: "#search", showSubResults: true});
});
</script>
<script type="speculationrules">
{
"prerender": [
{
"source": "document",
"where": {
"and": [
{"href_matches": "/*"}
]
},
"eagerness": "moderate"
}
]
}
</script>
</body>
</html>

257
blog/index.html Normal file
View file

@ -0,0 +1,257 @@
<!DOCTYPE html>
<html><head>
<meta charset="utf-8">
<title>Blog | Home</title>
<meta name="description" content="">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="mobile-web-app-capable" content="yes">
<meta name="color-scheme" content="dark light">
<link rel="stylesheet" href="/style.css">
<link rel="preload" href="/fonts/roboto_slab/RobotoSlab-VariableFont_wght.ttf" as="font" type="font/ttf" crossorigin />
<link rel="preload" href="/fonts/nyght-serif-main/fonts/WEB/NyghtSerif-Regular.woff2" as="font" type="font/woff2" crossorigin />
<link rel="preload" href="/fonts/nyght-serif-main/fonts/WEB/NyghtSerif-Bold.woff2" as="font" type="font/woff2" crossorigin />
<link rel="preload" href="/fonts/nyght-serif-main/fonts/WEB/NyghtSerif-BoldItalic.woff2" as="font" type="font/woff2" crossorigin />
</head>
<body><nav>
<div class="internal">
<a href="/" title="Home">Home</a>
<a href="/blog/" title="Blog" class="active">Blog</a>
<a href="/projects/" title="Projects">Projects</a>
<a href="/about/" title="About">About</a>
</div>
<div class="search">
<div id="search"></div>
</div>
<div class="external">
<a href="https://git.jaseg.de/" title="cgit">cgit</a>
<a href="https://github.com/jaseg" title="Github">Github</a>
<a href="https://gitlab.com/neinseg" title="Gitlab">Gitlab</a>
<a href="https://chaos.social/@jaseg" title="Mastodon">Mastodon</a>
</span>
</nav>
<header>
<h1>Blog</h1>
<ul class="breadcrumbs">
<li><a href="/">jaseg.de</a></li><li><a href="/blog/">Blog</a></li>
</ul>
</header>
<main class="cards">
<div class="card"><h3><a href="/blog/paper-sampling-mesh-monitor/">New paper: Monitoring Tamper-Sensing Meshes Using Low-Cost, Embedded Time-Domain Reflectometry</a></h3><strong>2025-10-20</strong>
<div class="summary">
<div class="document">
<p>foobar</p>
</div>
<a href="http://jaseg.de/blog/paper-sampling-mesh-monitor/">Read more</a>
</div>
</div>
<div class="card"><h3><a href="/blog/wsdiff-static-html-diffs/">wsdiff: Responsive diffs in plain HTML</a></h3><strong>2025-07-25</strong>
<div class="summary">
<div class="document">
<p>There's many tools that render diffs on the web, but almost none that work well on small screens such as phones. I fixed this by publishing wsdiff, a diffing tool written in Python that produces diffs as beautiful, responsive, static, self-contained HTML pages. wsdiffs wrap text to fit the window, and dynamically switch between unified and split diffs based on screen size using only CSS.</p>
</div>
<a href="http://jaseg.de/blog/wsdiff-static-html-diffs/">Read more</a>
</div>
</div>
<div class="card"><h3><a href="/blog/css-only-code-blocks/">Code listings with nice line wrapping and line numbers from plain CSS</a></h3><strong>2025-07-23</strong>
<div class="summary">
<div class="document">
<p>Code listings in web pages are often a bit of a pain to use. Usually, they don't wrap on small screens. Also, copy-pasting code from a code listing often copies the line numbers along with the code. Finally, many implementations use heavyweight HTML and/or javascript, making them slow to render. For this blog, I wrote a little CSS hack that renders nice, wrapping code blocks with line continuation markers in plain CSS without any JS.</p>
</div>
<a href="http://jaseg.de/blog/css-only-code-blocks/">Read more</a>
</div>
</div>
<div class="card"><h3><a href="/blog/jupyterlab-notebook-file-oneliner/">Getting the .ipynb Notebook File Location From a Running Jupyter Lab Notebook</a></h3><strong>2025-06-29</strong>
<div class="summary">
<div class="document">
<p>If you need to get the path of the ipynb file in a running #Jupyter notebook, this one-liner will do the trick. It seems chatgpt is confused, and a bunch of other approaches on the web look fragile and/or unnecessarily complex to me.</p>
</div>
<a href="http://jaseg.de/blog/jupyterlab-notebook-file-oneliner/">Read more</a>
</div>
</div>
<div class="card"><h3><a href="/blog/8seg/">8seg Technical Overview</a></h3><strong>2023-12-26</strong>
<div class="summary">
<div class="document">
<p>8seg is a large-scale LED light art installation that displays text on a 1.5 meter high, 30 meter wide 8-segment display made from cheap LED tape.</p>
</div>
<a href="http://jaseg.de/blog/8seg/">Read more</a>
</div>
</div>
<div class="card"><h3><a href="/blog/telekom-gpon-sfp/">Ubiquiti EdgeRouter on Deutsche Telekom GPON Fiber</a></h3><strong>2022-02-21</strong>
<div class="summary">
<div class="document">
<p>Short tutorial on getting a Deutsche Telekom GPON internet connection running using a SFP ONU unit in an Ubiquiti EdgeRouter.</p>
</div>
<a href="http://jaseg.de/blog/telekom-gpon-sfp/">Read more</a>
</div>
</div>
<div class="card"><h3><a href="/blog/ihsm-worlds-first-diy-hsm/">New Paper on Inertial Hardware Security Modules</a></h3><strong>2021-11-23</strong>
<div class="summary">
<div class="document">
<p>Paper announcement: We have published a paper on how you can DIY a tamper-sensing hardware security module from any single-board computer using a moving tamper-sensing mesh made from cheap PCBs.</p>
</div>
<a href="http://jaseg.de/blog/ihsm-worlds-first-diy-hsm/">Read more</a>
</div>
</div>
<div class="card"><h3><a href="/blog/kicad-mesh-plugin/">Kicad Mesh Plugin</a></h3><strong>2020-08-18</strong>
<div class="summary">
<div class="document">
<p>I wrote a little KiCad plugin that you can use to create security meshes, heaters and other things where you need one or more traces cover the entire surface of a PCB. The plugin supports arbitrary PCB shapes, cutouts, and can route around existing footprints and traces on the PCB.</p>
</div>
<a href="http://jaseg.de/blog/kicad-mesh-plugin/">Read more</a>
</div>
</div>
<div class="card"><h3><a href="/blog/private-contact-discovery/">Private Contact Discovery</a></h3><strong>2019-06-22</strong>
<div class="summary">
<div class="document">
<p>I gave a short introduction into Private Contact Discovery protocols at our university workgroup.</p>
</div>
<a href="http://jaseg.de/blog/private-contact-discovery/">Read more</a>
</div>
</div>
<div class="card"><h3><a href="/blog/hsm-basics/">Hardware Security Module Basics</a></h3><strong>2019-05-17</strong>
<div class="summary">
<div class="document">
<p>I gave a short introduction into Hardware Security Modules at our university workgroup, including an overview on interesting research directions.</p>
</div>
<a href="http://jaseg.de/blog/hsm-basics/">Read more</a>
</div>
</div>
<div class="card"><h3><a href="/blog/serial-protocols/">How to talk to your microcontroller over serial</a></h3><strong>2018-05-19</strong>
<div class="summary">
<div class="document">
<p>Scroll to the end for the <a class="reference internal" href="#conclusion">TL;DR</a>.</p>
<p>In this article I will give an overview on the protocols spoken on serial ports, highlighting common pitfalls. I will
summarize some points on how to design a serial protocol that is simple to implement and works reliably even under error
conditions.</p>
<p>If you have done low-level microcontroller firmware you will regularly have had to stuff some data up a serial port to
another microcontroller or to a computer. In the age of USB, an old-school serial port is still the simplest and
quickest way to get communication to a control computer up and running. Integrating a ten thousand-line USB stack into
your firmware and writing the necessary low-level drivers on the host side might take days. Poking a few registers to
set up your UART to talk to an external hardware USB to serial converter is a matter of minutes.</p></div>
<a href="http://jaseg.de/blog/serial-protocols/">Read more</a>
</div>
</div>
<div class="card"><h3><a href="/blog/thors-hammer/">Thor&#39;s Hammer</a></h3><strong>2018-05-03</strong>
<div class="summary">
<div class="document">
<p>In case you were having an inferiority complex because your friends' IBM Model M keyboards are so much louder than the shitty rubber dome freebie you got with your pc... Here's the solution: Thor's Hammer, a simple typing cadence enhancer for PS/2 keyboards.</p>
</div>
<a href="http://jaseg.de/blog/thors-hammer/">Read more</a>
</div>
</div>
<div class="card"><h3><a href="/blog/multichannel-led-driver/">32-Channel LED tape driver</a></h3><strong>2018-05-02</strong>
<div class="summary">
<div class="document">
<p>Together, a friend and I outfitted the small staircase at Berlin's Chaos Computer Club with nice, shiny RGB-WW LED tape for ambient lighting. For this installation, I made a 32-channel LED driver that achieves high dynamic range on all 32 channels using a cheap microcontroller by using Binary Code Modulation.</p>
</div>
<a href="http://jaseg.de/blog/multichannel-led-driver/">Read more</a>
</div>
</div>
<div class="card"><h3><a href="/blog/wifi-led-driver/">Wifi Led Driver</a></h3><strong>2018-05-02</strong>
<div class="summary">
<div class="document">
<p>After the multichannel LED driver was completed, I was just getting used to controlling LEDs at 14-bit resolution. I liked the board we designed in this project, but at 32 channels it was a bit large for most use cases. Sometimes I just want to pop a piece of LED tape or two somewhere, but I don't need a full 32 channels of control. I ended up thinking that a smaller version of the 32-channel driver that didn't require a separate control computer would be handy. So I sat down and designed a variant of the design with only 8 channels instead of 32 and an on-board ESP8266 module instead of the RS485 transceiver for WiFi connectivity.</p>
</div>
<a href="http://jaseg.de/blog/wifi-led-driver/">Read more</a>
</div>
</div>
<div class="card"><h3><a href="/blog/led-characterization/">LED Characterization</a></h3><strong>2018-05-02</strong>
<div class="summary">
<div class="document">
<p>Recently, I have been working on a small driver for ambient lighting using 12V LED strips like you can get inexpensively from China. I wanted to be able to just throw one of these somewhere, stick down some LED tape, hook it up to a small transformer and be able to control it through Wifi. When I was writing the firmware, I noticed that when fading between different colors, the colors look <em>all wrong</em>! This observation led me down a rabbit hole of color perception and LED peculiarities.</p>
</div>
<a href="http://jaseg.de/blog/led-characterization/">Read more</a>
</div>
</div>
</main><footer>
Copyright © 2025 Jan Sebastian Götte
/ <a href="/about/">About</a>
/ <a href="/imprint/">Imprint</a>
</footer>
<script type="text/javascript" src="/pagefind/pagefind-ui.js" defer></script>
<script>
window.addEventListener('DOMContentLoaded', (event) => {
new PagefindUI({element: "#search", showSubResults: true});
});
</script>
<script type="speculationrules">
{
"prerender": [
{
"source": "document",
"where": {
"and": [
{"href_matches": "/*"}
]
},
"eagerness": "moderate"
}
]
}
</script>
</body>
</html>

118
blog/index.xml Normal file
View file

@ -0,0 +1,118 @@
<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
<channel>
<title>Blog on Home</title>
<link>http://jaseg.de/blog/</link>
<description>Recent content in Blog on Home</description>
<generator>Hugo</generator>
<language>en-us</language>
<copyright>Jan Sebastian Götte</copyright>
<lastBuildDate>Mon, 20 Oct 2025 23:42:00 +0100</lastBuildDate>
<atom:link href="http://jaseg.de/blog/index.xml" rel="self" type="application/rss+xml" />
<item>
<title>New paper: Monitoring Tamper-Sensing Meshes Using Low-Cost, Embedded Time-Domain Reflectometry</title>
<link>http://jaseg.de/blog/paper-sampling-mesh-monitor/</link>
<pubDate>Mon, 20 Oct 2025 23:42:00 +0100</pubDate>
<guid>http://jaseg.de/blog/paper-sampling-mesh-monitor/</guid>
<description>&lt;div class=&#34;document&#34;&gt;&#xA;&#xA;&#xA;&lt;p&gt;foobar&lt;/p&gt;&#xA;&lt;/div&gt;</description>
</item>
<item>
<title>wsdiff: Responsive diffs in plain HTML</title>
<link>http://jaseg.de/blog/wsdiff-static-html-diffs/</link>
<pubDate>Fri, 25 Jul 2025 23:42:00 +0100</pubDate>
<guid>http://jaseg.de/blog/wsdiff-static-html-diffs/</guid>
<description>&lt;div class=&#34;document&#34;&gt;&#xA;&#xA;&#xA;&lt;p&gt;There&#39;s many tools that render diffs on the web, but almost none that work well on small screens such as phones. I fixed this by publishing wsdiff, a diffing tool written in Python that produces diffs as beautiful, responsive, static, self-contained HTML pages. wsdiffs wrap text to fit the window, and dynamically switch between unified and split diffs based on screen size using only CSS.&lt;/p&gt;&#xA;&lt;/div&gt;</description>
</item>
<item>
<title>Code listings with nice line wrapping and line numbers from plain CSS</title>
<link>http://jaseg.de/blog/css-only-code-blocks/</link>
<pubDate>Wed, 23 Jul 2025 23:42:00 +0100</pubDate>
<guid>http://jaseg.de/blog/css-only-code-blocks/</guid>
<description>&lt;div class=&#34;document&#34;&gt;&#xA;&#xA;&#xA;&lt;p&gt;Code listings in web pages are often a bit of a pain to use. Usually, they don&#39;t wrap on small screens. Also, copy-pasting code from a code listing often copies the line numbers along with the code. Finally, many implementations use heavyweight HTML and/or javascript, making them slow to render. For this blog, I wrote a little CSS hack that renders nice, wrapping code blocks with line continuation markers in plain CSS without any JS.&lt;/p&gt;&#xA;&lt;/div&gt;</description>
</item>
<item>
<title>Getting the .ipynb Notebook File Location From a Running Jupyter Lab Notebook</title>
<link>http://jaseg.de/blog/jupyterlab-notebook-file-oneliner/</link>
<pubDate>Sun, 29 Jun 2025 23:42:00 +0100</pubDate>
<guid>http://jaseg.de/blog/jupyterlab-notebook-file-oneliner/</guid>
<description>&lt;div class=&#34;document&#34;&gt;&#xA;&#xA;&#xA;&lt;p&gt;If you need to get the path of the ipynb file in a running #Jupyter notebook, this one-liner will do the trick. It seems chatgpt is confused, and a bunch of other approaches on the web look fragile and/or unnecessarily complex to me.&lt;/p&gt;&#xA;&lt;/div&gt;</description>
</item>
<item>
<title>8seg Technical Overview</title>
<link>http://jaseg.de/blog/8seg/</link>
<pubDate>Tue, 26 Dec 2023 15:26:00 +0100</pubDate>
<guid>http://jaseg.de/blog/8seg/</guid>
<description>&lt;div class=&#34;document&#34;&gt;&#xA;&#xA;&#xA;&lt;p&gt;8seg is a large-scale LED light art installation that displays text on a 1.5 meter high, 30 meter wide 8-segment display made from cheap LED tape.&lt;/p&gt;&#xA;&lt;/div&gt;</description>
</item>
<item>
<title>Ubiquiti EdgeRouter on Deutsche Telekom GPON Fiber</title>
<link>http://jaseg.de/blog/telekom-gpon-sfp/</link>
<pubDate>Mon, 21 Feb 2022 20:00:00 +0100</pubDate>
<guid>http://jaseg.de/blog/telekom-gpon-sfp/</guid>
<description>&lt;div class=&#34;document&#34;&gt;&#xA;&#xA;&#xA;&lt;p&gt;Short tutorial on getting a Deutsche Telekom GPON internet connection running using a SFP ONU unit in an Ubiquiti EdgeRouter.&lt;/p&gt;&#xA;&lt;/div&gt;</description>
</item>
<item>
<title>New Paper on Inertial Hardware Security Modules</title>
<link>http://jaseg.de/blog/ihsm-worlds-first-diy-hsm/</link>
<pubDate>Tue, 23 Nov 2021 23:42:20 +0100</pubDate>
<guid>http://jaseg.de/blog/ihsm-worlds-first-diy-hsm/</guid>
<description>&lt;div class=&#34;document&#34;&gt;&#xA;&#xA;&#xA;&lt;p&gt;Paper announcement: We have published a paper on how you can DIY a tamper-sensing hardware security module from any single-board computer using a moving tamper-sensing mesh made from cheap PCBs.&lt;/p&gt;&#xA;&lt;/div&gt;</description>
</item>
<item>
<title>Kicad Mesh Plugin</title>
<link>http://jaseg.de/blog/kicad-mesh-plugin/</link>
<pubDate>Tue, 18 Aug 2020 13:15:39 +0200</pubDate>
<guid>http://jaseg.de/blog/kicad-mesh-plugin/</guid>
<description>&lt;div class=&#34;document&#34;&gt;&#xA;&#xA;&#xA;&lt;p&gt;I wrote a little KiCad plugin that you can use to create security meshes, heaters and other things where you need one or more traces cover the entire surface of a PCB. The plugin supports arbitrary PCB shapes, cutouts, and can route around existing footprints and traces on the PCB.&lt;/p&gt;&#xA;&lt;/div&gt;</description>
</item>
<item>
<title>Private Contact Discovery</title>
<link>http://jaseg.de/blog/private-contact-discovery/</link>
<pubDate>Sat, 22 Jun 2019 10:30:00 +0800</pubDate>
<guid>http://jaseg.de/blog/private-contact-discovery/</guid>
<description>&lt;div class=&#34;document&#34;&gt;&#xA;&#xA;&#xA;&lt;p&gt;I gave a short introduction into Private Contact Discovery protocols at our university workgroup.&lt;/p&gt;&#xA;&lt;/div&gt;</description>
</item>
<item>
<title>Hardware Security Module Basics</title>
<link>http://jaseg.de/blog/hsm-basics/</link>
<pubDate>Fri, 17 May 2019 15:29:20 +0800</pubDate>
<guid>http://jaseg.de/blog/hsm-basics/</guid>
<description>&lt;div class=&#34;document&#34;&gt;&#xA;&#xA;&#xA;&lt;p&gt;I gave a short introduction into Hardware Security Modules at our university workgroup, including an overview on interesting research directions.&lt;/p&gt;&#xA;&lt;/div&gt;</description>
</item>
<item>
<title>How to talk to your microcontroller over serial</title>
<link>http://jaseg.de/blog/serial-protocols/</link>
<pubDate>Sat, 19 May 2018 08:09:46 +0200</pubDate>
<guid>http://jaseg.de/blog/serial-protocols/</guid>
<description>&lt;div class=&#34;document&#34;&gt;&#xA;&#xA;&#xA;&lt;p&gt;Scroll to the end for the &lt;a class=&#34;reference internal&#34; href=&#34;#conclusion&#34;&gt;TL;DR&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;In this article I will give an overview on the protocols spoken on serial ports, highlighting common pitfalls. I will&#xA;summarize some points on how to design a serial protocol that is simple to implement and works reliably even under error&#xA;conditions.&lt;/p&gt;&#xA;&lt;p&gt;If you have done low-level microcontroller firmware you will regularly have had to stuff some data up a serial port to&#xA;another microcontroller or to a computer. In the age of USB, an old-school serial port is still the simplest and&#xA;quickest way to get communication to a control computer up and running. Integrating a ten thousand-line USB stack into&#xA;your firmware and writing the necessary low-level drivers on the host side might take days. Poking a few registers to&#xA;set up your UART to talk to an external hardware USB to serial converter is a matter of minutes.&lt;/p&gt;&lt;/div&gt;</description>
</item>
<item>
<title>Thor&#39;s Hammer</title>
<link>http://jaseg.de/blog/thors-hammer/</link>
<pubDate>Thu, 03 May 2018 11:59:37 +0200</pubDate>
<guid>http://jaseg.de/blog/thors-hammer/</guid>
<description>&lt;div class=&#34;document&#34;&gt;&#xA;&#xA;&#xA;&lt;p&gt;In case you were having an inferiority complex because your friends&#39; IBM Model M keyboards are so much louder than the shitty rubber dome freebie you got with your pc... Here&#39;s the solution: Thor&#39;s Hammer, a simple typing cadence enhancer for PS/2 keyboards.&lt;/p&gt;&#xA;&lt;/div&gt;</description>
</item>
<item>
<title>32-Channel LED tape driver</title>
<link>http://jaseg.de/blog/multichannel-led-driver/</link>
<pubDate>Wed, 02 May 2018 11:31:14 +0200</pubDate>
<guid>http://jaseg.de/blog/multichannel-led-driver/</guid>
<description>&lt;div class=&#34;document&#34;&gt;&#xA;&#xA;&#xA;&lt;p&gt;Together, a friend and I outfitted the small staircase at Berlin&#39;s Chaos Computer Club with nice, shiny RGB-WW LED tape for ambient lighting. For this installation, I made a 32-channel LED driver that achieves high dynamic range on all 32 channels using a cheap microcontroller by using Binary Code Modulation.&lt;/p&gt;&#xA;&lt;/div&gt;</description>
</item>
<item>
<title>Wifi Led Driver</title>
<link>http://jaseg.de/blog/wifi-led-driver/</link>
<pubDate>Wed, 02 May 2018 11:31:03 +0200</pubDate>
<guid>http://jaseg.de/blog/wifi-led-driver/</guid>
<description>&lt;div class=&#34;document&#34;&gt;&#xA;&#xA;&#xA;&lt;p&gt;After the multichannel LED driver was completed, I was just getting used to controlling LEDs at 14-bit resolution. I liked the board we designed in this project, but at 32 channels it was a bit large for most use cases. Sometimes I just want to pop a piece of LED tape or two somewhere, but I don&#39;t need a full 32 channels of control. I ended up thinking that a smaller version of the 32-channel driver that didn&#39;t require a separate control computer would be handy. So I sat down and designed a variant of the design with only 8 channels instead of 32 and an on-board ESP8266 module instead of the RS485 transceiver for WiFi connectivity.&lt;/p&gt;&#xA;&lt;/div&gt;</description>
</item>
<item>
<title>LED Characterization</title>
<link>http://jaseg.de/blog/led-characterization/</link>
<pubDate>Wed, 02 May 2018 11:18:38 +0200</pubDate>
<guid>http://jaseg.de/blog/led-characterization/</guid>
<description>&lt;div class=&#34;document&#34;&gt;&#xA;&#xA;&#xA;&lt;p&gt;Recently, I have been working on a small driver for ambient lighting using 12V LED strips like you can get inexpensively from China. I wanted to be able to just throw one of these somewhere, stick down some LED tape, hook it up to a small transformer and be able to control it through Wifi. When I was writing the firmware, I noticed that when fading between different colors, the colors look &lt;em&gt;all wrong&lt;/em&gt;! This observation led me down a rabbit hole of color perception and LED peculiarities.&lt;/p&gt;&#xA;&lt;/div&gt;</description>
</item>
</channel>
</rss>

85
blog/jupyterlab-notebook-file-oneliner/index.html Normal file
View file

@ -0,0 +1,85 @@
<!DOCTYPE html>
<html><head>
<meta charset="utf-8">
<title>Getting the .ipynb Notebook File Location From a Running Jupyter Lab Notebook | Home</title>
<meta name="description" content="">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="mobile-web-app-capable" content="yes">
<meta name="color-scheme" content="dark light">
<link rel="stylesheet" href="/style.css">
<link rel="preload" href="/fonts/roboto_slab/RobotoSlab-VariableFont_wght.ttf" as="font" type="font/ttf" crossorigin />
<link rel="preload" href="/fonts/nyght-serif-main/fonts/WEB/NyghtSerif-Regular.woff2" as="font" type="font/woff2" crossorigin />
<link rel="preload" href="/fonts/nyght-serif-main/fonts/WEB/NyghtSerif-Bold.woff2" as="font" type="font/woff2" crossorigin />
<link rel="preload" href="/fonts/nyght-serif-main/fonts/WEB/NyghtSerif-BoldItalic.woff2" as="font" type="font/woff2" crossorigin />
</head>
<body><nav>
<div class="internal">
<a href="/" title="Home">Home</a>
<a href="/blog/" title="Blog">Blog</a>
<a href="/projects/" title="Projects">Projects</a>
<a href="/about/" title="About">About</a>
</div>
<div class="search">
<div id="search"></div>
</div>
<div class="external">
<a href="https://git.jaseg.de/" title="cgit">cgit</a>
<a href="https://github.com/jaseg" title="Github">Github</a>
<a href="https://gitlab.com/neinseg" title="Gitlab">Gitlab</a>
<a href="https://chaos.social/@jaseg" title="Mastodon">Mastodon</a>
</span>
</nav>
<header>
<h1>Getting the .ipynb Notebook File Location From a Running Jupyter Lab Notebook</h1>
<ul class="breadcrumbs">
<li><a href="/">jaseg.de</a></li>
<li><a href="/blog/">Blog</a></li><li><a href="/blog/jupyterlab-notebook-file-oneliner/">Getting the .ipynb Notebook File Location From a Running Jupyter Lab Notebook</a></li>
</ul>
<strong>2025-06-29</strong>
</header>
<main data-pagefind-body>
<div class="document">
<p>If you need to get the path of the ipynb file in a running #Jupyter notebook, this one-liner will do the trick. It seems
chatgpt is confused, and a bunch of other approaches on the web look fragile and/or unnecessarily complex to me.</p>
<pre class="code python literal-block">
<span class="lineno"></span><span class="line"><span class="kn">import</span><span class="w"> </span><span class="nn">sys</span><span class="w"></span></span>
<span class="lineno"></span><span class="line"><span class="w"></span><span class="n">Path</span><span class="p">(</span><span class="n">json</span><span class="o">.</span><span class="n">loads</span><span class="p">(</span><span class="n">Path</span><span class="p">(</span><span class="n">sys</span><span class="o">.</span><span class="n">argv</span><span class="p">[</span><span class="o">-</span><span class="mi">1</span><span class="p">])</span><span class="o">.</span><span class="n">read_bytes</span><span class="p">())[</span><span class="s1">'jupyter_session'</span><span class="p">])</span>
</span></pre>
<p>The way this works is that for each notebook, jupyter starts a python &quot;kernel&quot; process that actually runs the notebook's
code. That kernel gets a json file with info on the notebook's location on the disk passed through its command line.
Since we're running code in that exact python process, we can just grab that json file from sys.argv, and read it
ourselves.</p>
</div>
</main><footer>
Copyright © 2025 Jan Sebastian Götte
/ <a href="/about/">About</a>
/ <a href="/imprint/">Imprint</a>
</footer>
<script type="text/javascript" src="/pagefind/pagefind-ui.js" defer></script>
<script>
window.addEventListener('DOMContentLoaded', (event) => {
new PagefindUI({element: "#search", showSubResults: true});
});
</script>
<script type="speculationrules">
{
"prerender": [
{
"source": "document",
"where": {
"and": [
{"href_matches": "/*"}
]
},
"eagerness": "moderate"
}
]
}
</script>
</body>
</html>

0
content/blog/kicad-mesh-plugin/images/anim.webp → blog/kicad-mesh-plugin/images/anim.webp Executable file → Normal file
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 35 KiB

After

Width:  |  Height:  |  Size: 35 KiB

Before After
Before After

0
content/blog/kicad-mesh-plugin/images/cells-0.svg → blog/kicad-mesh-plugin/images/cells-0.svg Executable file → Normal file
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 292 KiB

After

Width:  |  Height:  |  Size: 292 KiB

Before After
Before After

0
content/blog/kicad-mesh-plugin/images/cells-100.svg → blog/kicad-mesh-plugin/images/cells-100.svg Executable file → Normal file
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 296 KiB

After

Width:  |  Height:  |  Size: 296 KiB

Before After
Before After

0
content/blog/kicad-mesh-plugin/images/cells-25.svg → blog/kicad-mesh-plugin/images/cells-25.svg Executable file → Normal file
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 296 KiB

After

Width:  |  Height:  |  Size: 296 KiB

Before After
Before After

0
content/blog/kicad-mesh-plugin/images/cells-50.svg → blog/kicad-mesh-plugin/images/cells-50.svg Executable file → Normal file
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 296 KiB

After

Width:  |  Height:  |  Size: 296 KiB

Before After
Before After

0
content/blog/kicad-mesh-plugin/images/cells-75.svg → blog/kicad-mesh-plugin/images/cells-75.svg Executable file → Normal file
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 296 KiB

After

Width:  |  Height:  |  Size: 296 KiB

Before After
Before After

0
content/blog/kicad-mesh-plugin/images/grid-vis-plain.svg → blog/kicad-mesh-plugin/images/grid-vis-plain.svg Executable file → Normal file
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 46 KiB

After

Width:  |  Height:  |  Size: 46 KiB

Before After
Before After

0
content/blog/kicad-mesh-plugin/images/grid-vis.svg → blog/kicad-mesh-plugin/images/grid-vis.svg Executable file → Normal file
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 46 KiB

After

Width:  |  Height:  |  Size: 46 KiB

Before After
Before After

0
content/blog/kicad-mesh-plugin/images/kicad-mesh-outline.png → blog/kicad-mesh-plugin/images/kicad-mesh-outline.png Executable file → Normal file
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 118 KiB

After

Width:  |  Height:  |  Size: 118 KiB

Before After
Before After

0
content/blog/kicad-mesh-plugin/images/kicad-mesh-result-large.png → blog/kicad-mesh-plugin/images/kicad-mesh-result-large.png Executable file → Normal file
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 193 KiB

After

Width:  |  Height:  |  Size: 193 KiB

Before After
Before After

0
content/blog/kicad-mesh-plugin/images/kicad-mesh-settings.png → blog/kicad-mesh-plugin/images/kicad-mesh-settings.png Executable file → Normal file
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 45 KiB

After

Width:  |  Height:  |  Size: 45 KiB

Before After
Before After

0
content/blog/kicad-mesh-plugin/images/kicad-mesh-settings2.png → blog/kicad-mesh-plugin/images/kicad-mesh-settings2.png Executable file → Normal file
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 128 KiB

After

Width:  |  Height:  |  Size: 128 KiB

Before After
Before After

0
content/blog/kicad-mesh-plugin/images/maze_tiles.svg → blog/kicad-mesh-plugin/images/maze_tiles.svg Executable file → Normal file
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 189 KiB

After

Width:  |  Height:  |  Size: 189 KiB

Before After
Before After

0
content/blog/kicad-mesh-plugin/images/maze_tiles_plain.svg → blog/kicad-mesh-plugin/images/maze_tiles_plain.svg Executable file → Normal file
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 175 KiB

After

Width:  |  Height:  |  Size: 175 KiB

Before After
Before After

0
content/blog/kicad-mesh-plugin/images/modern_art.svg → blog/kicad-mesh-plugin/images/modern_art.svg Executable file → Normal file
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 104 KiB

After

Width:  |  Height:  |  Size: 104 KiB

Before After
Before After

0
content/blog/kicad-mesh-plugin/images/tiles-25-small.svg → blog/kicad-mesh-plugin/images/tiles-25-small.svg Executable file → Normal file
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 1.7 MiB

After

Width:  |  Height:  |  Size: 1.7 MiB

Before After
Before After

0
content/blog/kicad-mesh-plugin/images/traces-25-small.svg → blog/kicad-mesh-plugin/images/traces-25-small.svg Executable file → Normal file
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 809 KiB

After

Width:  |  Height:  |  Size: 809 KiB

Before After
Before After

238
blog/kicad-mesh-plugin/index.html Normal file
View file

@ -0,0 +1,238 @@
<!DOCTYPE html>
<html><head>
<meta charset="utf-8">
<title>Kicad Mesh Plugin | Home</title>
<meta name="description" content="">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="mobile-web-app-capable" content="yes">
<meta name="color-scheme" content="dark light">
<link rel="stylesheet" href="/style.css">
<link rel="preload" href="/fonts/roboto_slab/RobotoSlab-VariableFont_wght.ttf" as="font" type="font/ttf" crossorigin />
<link rel="preload" href="/fonts/nyght-serif-main/fonts/WEB/NyghtSerif-Regular.woff2" as="font" type="font/woff2" crossorigin />
<link rel="preload" href="/fonts/nyght-serif-main/fonts/WEB/NyghtSerif-Bold.woff2" as="font" type="font/woff2" crossorigin />
<link rel="preload" href="/fonts/nyght-serif-main/fonts/WEB/NyghtSerif-BoldItalic.woff2" as="font" type="font/woff2" crossorigin />
</head>
<body><nav>
<div class="internal">
<a href="/" title="Home">Home</a>
<a href="/blog/" title="Blog">Blog</a>
<a href="/projects/" title="Projects">Projects</a>
<a href="/about/" title="About">About</a>
</div>
<div class="search">
<div id="search"></div>
</div>
<div class="external">
<a href="https://git.jaseg.de/" title="cgit">cgit</a>
<a href="https://github.com/jaseg" title="Github">Github</a>
<a href="https://gitlab.com/neinseg" title="Gitlab">Gitlab</a>
<a href="https://chaos.social/@jaseg" title="Mastodon">Mastodon</a>
</span>
</nav>
<header>
<h1>Kicad Mesh Plugin</h1>
<ul class="breadcrumbs">
<li><a href="/">jaseg.de</a></li>
<li><a href="/blog/">Blog</a></li><li><a href="/blog/kicad-mesh-plugin/">Kicad Mesh Plugin</a></li>
</ul>
<strong>2020-08-18</strong>
</header>
<main data-pagefind-body>
<div class="document">
<figure data-pagefind-ignore>
<img src="images/anim.webp" style="max-width: 20em">
</figure><div class="section" id="tamper-detection-meshes">
<h2>Tamper Detection Meshes</h2>
<p>Cryptography is at the foundation of our modern, networked world. From email to card payment infrastructure in brick and
mortar stores, cryptographic keys secure almost every part of our digital lives againts cybercriminals or curious
surveillance capitalists. Without cryptography, many of the things we routinely do in our lives such as paying for
groceries with a credit card, messaging a friend on <a class="reference external" href="https://signal.org">Signal</a> or unlocking a car with its keyfob
would not be possible. The security of all of these systems in its core lies on the secrecy of cryptographic keys.
Systems differ in what kind of keys they use, how often these keys are replaced and the intricacies of the cryptographic
operations these keys fit into but all have in common that their security relies on keeping the keys secret.</p>
<p>In practice, this secrecy has been implemented in many different ways. Mass-market software such as browsers or
messenger apps usually relies on some operating system facility to tell the computer &quot;<em>please keep this piece of memory
away from all other applications</em>&quot;. While on desktop operating systems usually this does not provide much of a barrier
to other programs on the same computer, on modern mobile operating systems this approach is actually quite secure.
However, given sufficient resources no security is perfect. All of these systems can be compromised if the host
operating system is compromised sufficiently, and for organizations with considerable resources a market has sprung up
that offers turn-key solutions for all wiretapping needs.</p>
<p>In some applications, this level of security has not been considered sufficient. Particularly financial infrastructure
is such a high-profile target that a lot of effort has been put into the security of cryptographic implementations. The
best cryptographic algorithm is useless if it is run on a compromised system (from that system's point of view anyway).
One of the core cryptographic components in financial applications are smartcards like they are used as payment cards in
most countries nowadays. These smartcards contain a small, specialized cryptographic microcontroller that is designed to
be hard to tamper with. Though one of the design goals of the system is to reduce the amount of sensitive information
stored on the card, things such as copying of a card can only be hindered by making the chip hard to read out.</p>
<figure data-pagefind-ignore>
<img src="images/modern_art.svg" style="max-width: 20em">
</figure><p>With smartcards being the means of choice on one side of the counter in electronic payments, on the other side of the
counter a different technology prevails. Attacks on payment terminals are bound to have much more dire consequences than
attacks on individual cards since one terminal might see hundreds of cards being read every day. For this reason, the
level of attack countermeasures employed in these terminals is a considerable step up from bare smartcards. While a
smartcard is made physically hard to tamper, it does not have a battery and it can only detect tampering once it is
powered by a reader. This allows for well-equipped attackers to use tools such as Focused Ion Beam (FIB) workstations to
circumvent the smartcard's defences while it is powered down, and then power up the card to carry out the actual attack.</p>
<p>The answer to this problem in electronic payment infrastructure is called <em>Hardware Security Module</em>, or HSM. An HSM is
similar to a smartcard in its function (cryptographic processing using keys that are meant to never leave the protection
of the HSM). The one major between the two is that an HSM has its own battery and is continuously powered from its
manufacture to the day it is scrapped. If the HSM looses power at any point in time, it uses a small amount of energy
stored internally to securely wipe all cryptographic secrets from its memory within a few milliseconds.</p>
<p>Being powered at all times allows the HSM to actively detect and respond to attacks. The most common way this is done is
by wrapping the juicy secret parts in a foil or a printed circuit board that is patterned with a long and convoluted
maze of wires, called a <em>mesh</em>. The HSM is continuously monitoring these wires for changes (such as shorts, breaks or
changes in resistance) and will sound the alarm when any are detected. Practically, this presents a considerable hurdle
to any attacker: They have to find a way to disable or circumvent the mesh while it is being monitored by the HSM. In
practice, often this is no insurmountable challenge but it again increases attack costs.</p>
</div>
<div class="section" id="diy-meshes">
<h2>DIY Meshes</h2>
<p>Throughout my studies in security research I have always had an interest in HSMs. I have taken apart my fair share of
HSMs and at this point, to understand the technology more, I want to experiment with building my own HSM. In last year's
<a class="reference external" href="http://jaseg.de/blog/hsm-basics/">HSM basics</a> post I have lined out some ideas for a next generation design that
deviates from the bread-and-butter apporoach of using a mesh as the primary security feature. Before embarking on
practical experiments with these ideas, I want to first take a stab at replicating the current state of the art as best
I can. State of the art meshes often use exotic substrates such as 3D plastic parts with traces chemically deposited on
their surface or special flexible substrates with conductive ink traces. These technologies will likely be too
cumbersome for me to implement myself only for a few prototypes, and industrial manufacturers will most likely be too
expensive. Thus, I will concentrate on regular PCB technology for now.</p>
<p>The idea of a mesh on a PCB is pretty simple: You have one or several traces that you try to cover every corner of the
mesh PCB's area with. To be most effective, the traces should be as thin and as close together as possible. To increase
the chances of a manipulation being detected, multiple traces can also be used that can then be monitored for shorts
between them.</p>
<p>While one can feasibly lay out these traces by hand, this really is an ideal application of a simple auto-router. While
general PCB autorouting is <em>hard</em>, auto-routing just a few traces to approximate a space-filling curve is not. Since I
am just starting out, I went with the simplest algorithmic solution I could think of. I first approximate the area
designated to the mesh with a square grid whose cells are a multiple of my trace/space size. The mesh will only be drawn
into grid cells that are fully inside the set boundaries. All cells outside or going across the border are discarded in
this step.</p>
<p>I decided to implement this auto-router in a KiCAD plugin. Though KiCADs plugin API is not the best, it was just about
usable for this task.</p>
<figure data-pagefind-ignore>
<img src="images/kicad-mesh-outline.png" alt="KiCAD showing an irregular board shape with rounded corners and
indents. In the middle of the board there is a footprint for a 4-pin surface-mount pin header.">
<figcaption>The process starts out with the mesh shape being defined inside KiCAD. The mesh's outline is drawn
onto one of the graphical "Eco" layers. A footprint is placed to serve as a placeholder for the mesh's
connections to the outside world. This footprint is later used as the starting point for the mesh generation
algorithm.</figcaption>
</figure><figure data-pagefind-ignore>
<img src="images/grid-vis-plain.svg" alt="A vizualization of the grid fitting process. Over the mesh's irregular
outline a grid is drawn. In this picture, all grid cells that are fully inside the grid are shown. Grid cells
that overlap the mesh border are highlighted. Grid cells outside of the mesh border are not drawn.">
<figcaption>A visualization of the grid fitting process. First, a grid large enough to contain the mesh border
is generated. Then, every cell is checked for overlap with the mesh border area. If the cell is fully inside, it
(yellow), it is considered in the mesh generation later. Cells outside (gray) or on the border (red) are
discarded.</figcaption>
</figure><p>After generating the grid, starting from the place I want to connect to the mesh, I walk the grid's cells one by one to
generate a tree that covers the entire grid's area. To set the mesh's starting place I place a footprint on the board
(dark gray in the picture above) whose designator I then tell my script. The tree generation algorithm looks like a
depth-first search, except all checks are random. Depending on the level of randomness used at each step of the
algorithm it yields more or less organized-looking results. Below are five example runs of the algorithm at differing
levels of randomness with the cells colored according to their distance from the tree root. 0% randomness means that the
algorithm is going to try cells in forward direction first on every step, and only then try out left and right. 100%
means that on every step, the algorithm is choosing a new direction at random.</p>
<div class="subfigure" data-pagefind-ignore>
<figure>
<img src="images/cells-0.svg" alt="a completely organized looking grid with spiral patterns all over.">
<figcaption>0%</figcaption>
</figure>
<figure>
<img src="images/cells-25.svg">
<figcaption>25%</figcaption>
</figure>
<figure>
<img src="images/cells-50.svg">
<figcaption>50%</figcaption>
</figure>
<figure>
<img src="images/cells-75.svg">
<figcaption>75%</figcaption>
</figure>
<figure>
<img src="images/cells-100.svg" alt="a completely random looking grid with cells aggregating into ireggular
areas that look like paint splotches.">
<figcaption>100%</figcaption>
</figure>
</div><p>After I have built this tree like you would do in a depth-first search, I draw my one or several mesh mesh traces into
it. The core observation here is that there is only 16 possible ways a cell can be connected: It has four neighbors,
each of which it can either be connected to or not, which results in 2^4 options. If you consider rotations and
mirroring, this works out to rotations or mirrored versions of only six base tiles: The empty tile, a tile with all four
sides connected, a straight through, a 90 degree bend, and a &quot;T&quot;-junction—see the illustration below.</p>
<figure data-pagefind-ignore>
<img src="images/maze_tiles_plain.svg" style="max-width: 20em">
<figcaption>
There are six possible tile types in our connectivity graph inside its square tiling. This graphic illustrates
all sixteen rotations of these with how they would look in a two-conductor mesh.
</figcaption>
</figure><p>After tiling the grid according to the key above, we get the result below.</p>
<figure data-pagefind-ignore>
<img src="images/tiles-25-small.svg">
<figcaption>
An auto-routed mesh with traces colored according to tile types.
</figcaption>
</figure><figure data-pagefind-ignore>
<img src="images/traces-25-small.svg">
<figcaption>
The same mesh, but with traces all black.
</figcaption>
</figure><p>Putting it all together got me the KiCAD plugin you can see in the screenshot below.</p>
<figure data-pagefind-ignore>
<img src="images/kicad-mesh-settings2.png">
<figcaption>
The plugin settings window open.
</figcaption>
</figure><figure data-pagefind-ignore>
<img src="images/kicad-mesh-result-large.png">
<figcaption>
After runing the plugin, the generated mesh looks like this in pcbnew.
</figcaption>
</figure><p>I am fairly happy with the result, but getting there was a medium pain. Especially KiCAD's plugin API is still very
unfinieshed. It is hard to use, most parts are completely undocumented and if you use anything but its most basic parts
things tend to break. One particular pain point for me was that after generating the mesh, the traces have been added to
the board, but are still invisible for some reason. You have to save the board first, then re-load the file for them to
become visible. Also KiCAD crashes whenever the plugin tries to remove a trace, so currently my workflow involves always
making a copy of the board file first and treating mesh generation as a non-reversible finishing step.</p>
<p><a class="reference external" href="https://git.jaseg.de/kimesh.git/tree/plugin/mesh_dialog.py">Check out the code on my cgit</a>.</p>
<!-- ::
.. raw:: html
<figure data-pagefind-ignore>
<img src="images/grid-vis-plain.svg" alt="">
<figcaption></figcaption>
</figure> -->
</div>
</div>
</main><footer>
Copyright © 2025 Jan Sebastian Götte
/ <a href="/about/">About</a>
/ <a href="/imprint/">Imprint</a>
</footer>
<script type="text/javascript" src="/pagefind/pagefind-ui.js" defer></script>
<script>
window.addEventListener('DOMContentLoaded', (event) => {
new PagefindUI({element: "#search", showSubResults: true});
});
</script>
<script type="speculationrules">
{
"prerender": [
{
"source": "document",
"where": {
"and": [
{"href_matches": "/*"}
]
},
"eagerness": "moderate"
}
]
}
</script>
</body>
</html>

0
content/blog/led-characterization/images/daylight_spectrum_dvd.jpg → blog/led-characterization/images/daylight_spectrum_dvd.jpg
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 77 KiB

After

Width:  |  Height:  |  Size: 77 KiB

Before After
Before After

0
content/blog/led-characterization/images/driver_ringing_strong.jpg → blog/led-characterization/images/driver_ringing_strong.jpg
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 279 KiB

After

Width:  |  Height:  |  Size: 279 KiB

Before After
Before After

0
content/blog/led-characterization/images/driver_ringing_weak.jpg → blog/led-characterization/images/driver_ringing_weak.jpg
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 285 KiB

After

Width:  |  Height:  |  Size: 285 KiB

Before After
Before After

0
content/blog/led-characterization/images/electronics_whole.jpg → blog/led-characterization/images/electronics_whole.jpg
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 428 KiB

After

Width:  |  Height:  |  Size: 428 KiB

Before After
Before After

0
content/blog/led-characterization/images/hsv_cylinder.png → blog/led-characterization/images/hsv_cylinder.png
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 287 KiB

After

Width:  |  Height:  |  Size: 287 KiB

Before After
Before After

0
content/blog/led-characterization/images/photodiode_sensitivity.svg → blog/led-characterization/images/photodiode_sensitivity.svg
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 29 KiB

After

Width:  |  Height:  |  Size: 29 KiB

Before After
Before After

0
content/blog/led-characterization/images/preamp_back.jpg → blog/led-characterization/images/preamp_back.jpg
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 332 KiB

After

Width:  |  Height:  |  Size: 332 KiB

Before After
Before After

0
content/blog/led-characterization/images/preamp_front.jpg → blog/led-characterization/images/preamp_front.jpg
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 301 KiB

After

Width:  |  Height:  |  Size: 301 KiB

Before After
Before After

0
content/blog/led-characterization/images/preamp_schematic.jpg → blog/led-characterization/images/preamp_schematic.jpg
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 271 KiB

After

Width:  |  Height:  |  Size: 271 KiB

Before After
Before After

0
content/blog/led-characterization/images/processed_plot_cheap_rgb.svg → blog/led-characterization/images/processed_plot_cheap_rgb.svg
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 88 KiB

After

Width:  |  Height:  |  Size: 88 KiB

Before After
Before After

0
content/blog/led-characterization/images/raw_plot_cheap_rgb.svg → blog/led-characterization/images/raw_plot_cheap_rgb.svg
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 103 KiB

After

Width:  |  Height:  |  Size: 103 KiB

Before After
Before After

0
content/blog/led-characterization/images/rgb_cube.svg → blog/led-characterization/images/rgb_cube.svg
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 17 KiB

After

Width:  |  Height:  |  Size: 17 KiB

Before After
Before After

0
content/blog/led-characterization/images/spectrograph_step1_parts.jpg → blog/led-characterization/images/spectrograph_step1_parts.jpg
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 278 KiB

After

Width:  |  Height:  |  Size: 278 KiB

Before After
Before After

0
content/blog/led-characterization/images/spectrograph_step2.jpg → blog/led-characterization/images/spectrograph_step2.jpg
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 271 KiB

After

Width:  |  Height:  |  Size: 271 KiB

Before After
Before After

0
content/blog/led-characterization/images/spectrograph_step3.jpg → blog/led-characterization/images/spectrograph_step3.jpg
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 250 KiB

After

Width:  |  Height:  |  Size: 250 KiB

Before After
Before After

0
content/blog/led-characterization/images/spectrograph_step4_complete.jpg → blog/led-characterization/images/spectrograph_step4_complete.jpg
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 297 KiB

After

Width:  |  Height:  |  Size: 297 KiB

Before After
Before After

0
content/blog/led-characterization/images/zeus_hammer_breadboard.jpg → blog/led-characterization/images/zeus_hammer_breadboard.jpg
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 435 KiB

After

Width:  |  Height:  |  Size: 435 KiB

Before After
Before After

0
content/blog/led-characterization/images/zeus_hammer_breadboard_original.jpg → blog/led-characterization/images/zeus_hammer_breadboard_original.jpg
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 2.4 MiB

After

Width:  |  Height:  |  Size: 2.4 MiB

Before After
Before After

0
content/blog/led-characterization/images/zeus_hammer_schematic.jpg → blog/led-characterization/images/zeus_hammer_schematic.jpg
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 145 KiB

After

Width:  |  Height:  |  Size: 145 KiB

Before After
Before After

0
content/blog/led-characterization/images/zeus_hammer_schematic_original.jpg → blog/led-characterization/images/zeus_hammer_schematic_original.jpg
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 1.8 MiB

After

Width:  |  Height:  |  Size: 1.8 MiB

Before After
Before After

452
blog/led-characterization/index.html Normal file
View file

@ -0,0 +1,452 @@
<!DOCTYPE html>
<html><head>
<meta charset="utf-8">
<title>LED Characterization | Home</title>
<meta name="description" content="">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="mobile-web-app-capable" content="yes">
<meta name="color-scheme" content="dark light">
<link rel="stylesheet" href="/style.css">
<link rel="preload" href="/fonts/roboto_slab/RobotoSlab-VariableFont_wght.ttf" as="font" type="font/ttf" crossorigin />
<link rel="preload" href="/fonts/nyght-serif-main/fonts/WEB/NyghtSerif-Regular.woff2" as="font" type="font/woff2" crossorigin />
<link rel="preload" href="/fonts/nyght-serif-main/fonts/WEB/NyghtSerif-Bold.woff2" as="font" type="font/woff2" crossorigin />
<link rel="preload" href="/fonts/nyght-serif-main/fonts/WEB/NyghtSerif-BoldItalic.woff2" as="font" type="font/woff2" crossorigin />
</head>
<body><nav>
<div class="internal">
<a href="/" title="Home">Home</a>
<a href="/blog/" title="Blog">Blog</a>
<a href="/projects/" title="Projects">Projects</a>
<a href="/about/" title="About">About</a>
</div>
<div class="search">
<div id="search"></div>
</div>
<div class="external">
<a href="https://git.jaseg.de/" title="cgit">cgit</a>
<a href="https://github.com/jaseg" title="Github">Github</a>
<a href="https://gitlab.com/neinseg" title="Gitlab">Gitlab</a>
<a href="https://chaos.social/@jaseg" title="Mastodon">Mastodon</a>
</span>
</nav>
<header>
<h1>LED Characterization</h1>
<ul class="breadcrumbs">
<li><a href="/">jaseg.de</a></li>
<li><a href="/blog/">Blog</a></li><li><a href="/blog/led-characterization/">LED Characterization</a></li>
</ul>
<strong>2018-05-02</strong>
</header>
<main data-pagefind-body>
<div class="document">
<div class="section" id="preface">
<h2>Preface</h2>
<p>Recently, I have been working on a <a class="reference external" href="http://jaseg.de/blog/wifi-led-driver/">small driver</a> for ambient lighting using 12V LED strips like you can get
inexpensively from China. I wanted to be able to just throw one of these somewhere, stick down some LED tape, hook it up
to a small transformer and be able to control it through Wifi. When I was writing the firmware, I noticed that when
fading between different colors, the colors look <em>all wrong</em>! This observation led me down a rabbit hole of color
perception and LED peculiarities.</p>
<p>The idea of the LED driver was that it can be used either with up to eight single-color LED tapes or, much more
interesting, with up to two RGB or RGBW (red-green-blue-white) LED tapes. For ambient lighting high color resolution was
really important so you could dim it down a lot without flickering. I ended up using the same driver stage I used in the
<a class="reference external" href="http://jaseg.de/blog/multichannel-led-driver/">multichannel LED driver</a> project for its great color resolution and low hardware requirements.</p>
<figure data-pagefind-ignore>
<img src="images/rgb_cube.svg" alt="An illustration of the RGB color cube.">
<figcaption>An illustration of the RGB color cube.
<a href="https://commons.wikimedia.org/wiki/File:RGB_color_cube.svg">Picture</a> by
<a href="https://commons.wikimedia.org/wiki/User:Maklaan">Maklaan from Wikimedia Commons</a>,
<a href="https://creativecommons.org/licenses/by-sa/3.0/">CC-BY-SA 3.0</a>
</figcaption>
</figure><p>To make setting colors over Wifi more intuitive I implemented support for HSV colors. RGB is fine for communication
between computers, but I think HSV is easier to work with when manually inputting colors from the command line. RGB is
close to how most monitors, cameras and the human visual apparatus work on a very low level but doesn't match
higher-level human color perception very well. When we describe a color we tend to think in terms of &quot;hue&quot; or
&quot;brightness&quot;, and computing a measure of those from RGB values is not easy.</p>
</div>
<div class="section" id="colors-and-color-spaces">
<h2>Colors and Color Spaces</h2>
<p><a class="reference external" href="https://en.wikipedia.org/wiki/Color_space">Color spaces</a> are a mathematical abstraction of the concept of color. When we say &quot;RGB&quot;, most of the time we actually
mean <a class="reference external" href="https://en.wikipedia.org/wiki/SRGB">sRGB</a>, a standardized notion of how to map three numbers labelled &quot;red&quot;, &quot;green&quot; and &quot;blue&quot; onto a perceived
color. <a class="reference external" href="https://en.wikipedia.org/wiki/HSL_and_HSV">HSV</a> is an early attempt to more closely align these numbers with our perception. After HSV, a number of other
<em>perceptual</em> color spaces such as <a class="reference external" href="https://en.wikipedia.org/wiki/CIE_1931_color_space">XYZ (CIE 1931)</a> and <a class="reference external" href="https://en.wikipedia.org/wiki/Lab_color_space">CIE Lab/LCh</a> were born, further improving this alignment. In
this mathematical model, mapping a color from one color space into another color space is just a coordinate
transformation.</p>
<figure data-pagefind-ignore>
<img src="images/hsv_cylinder.png" alt="An illustration of the HSV color space as a cylinder.">
<figcaption>An illustration of the HSV color space as a cylinder.
<a href="https://commons.wikimedia.org/wiki/File:HSV_color_solid_cylinder.png">Picture</a> by
<a href="https://commons.wikimedia.org/wiki/User:SharkD">SharkD from Wikimedia Commons</a>,
<a href="https://creativecommons.org/licenses/by-sa/3.0/">CC-BY-SA 3.0</a>
</figcaption>
</figure><p>CIE 1931 XYZ is much larger than any other color space, which is why it is a good basis to express other color spaces
in. In XYZ there are many coordinates that are outside of what the human eye can perceive. Below is an illustration of
the sRGB space within XYZ. The wireframe cube is (0,0,0) to (1,1,1) in XYZ. The colorful object in the middle is what
of sRGB fits inside XYZ, and the lines extending out from it indicate the space that can be expressed in sRGB but not in
XYZ. The fat white curve is a projection of the <em>monochromatic spectral locus</em>, that is the curve of points you get in
XYZ for pure visible wavelengths.</p>
<p>As you can see, sRGB is <em>much</em> smaller than XYZ or even the part within the monochromatic locus that we can perceive. In
particular in the blues and greens we loose <em>a lot</em> of colors to sRGB.</p>
<figure data-pagefind-ignore>
<video controls loop>
<source src="video/sRGB.mkv" type="video/h264">
<source src="video/sRGB.webm" type="video/webm">
Your browser does not support the HTML5 video tag.
</video>
<figcaption>Illustration of the measured sRGB color space within XYZ. The thick, white line is the spectral
locus.
<a href="video/sRGB.mkv">mkv/h264 download</a> /
<a href="video/sRGB.webm">webm download</a>
</figcaption>
</figure><p>The wrong colors I got when fading between colors were caused by this coordinate transformation being askew. Thinking
over the problem, there are several sources for imperfections:</p>
<ul class="simple">
<li>The LED driver may not be entirely linear. For most modulations such as PWM the brightness will be linear starting
from a certain value, but there is probably an offset caused by imperfect edges of the LED current. This offset can be
compensated with software calibration. I built a calibration setup for driver linearity in the <a class="reference external" href="http://jaseg.de/blog/multichannel-led-driver/">multichannel LED
driver</a> project. Below are pictures of ringing on the edges of an LED driver's waveform.</li>
<li>The red, green and blue channels of the LEDs used on the LED tape are not matched. This skews the RGB color space.
In practice, the blue channel of my RGB tape to me <em>looks</em> much brighter than the red channel.</li>
<li>The precise colors of the red, green and blue channels of the LEDs are unknown. Though the red channel <em>looks</em> red, it
may be of a slightly different hue compared to the reference red used in <a class="reference external" href="https://en.wikipedia.org/wiki/SRGB">sRGB</a> which would also skew the RGB color
space.</li>
</ul>
<div class="subfigure" data-pagefind-ignore>
<figure>
<img src="images/driver_ringing_strong.jpg" alt="Strong ringing on the LED voltage waveform edge at about
100% overshoot during about 70% of the cycle time.">
<figcaption>The LED strip being at the end of a couple meters of wire caused extremely bad ringing at high
driving frequencies.</figcaption>
</figure>
<figure>
<img src="images/driver_ringing_weak.jpg" alt="Weak ringing on the LED voltage waveform edge at about 30%
overshoot during about 20% of the cycle time.">
<figcaption>Adding a resistor in front of the MOSFET gate to slow the transition dampened the ringing
somewhat, but ultimately it cannot be eliminated entirely.</figcaption>
</figure>
</div><p>These last two errors are tricky to compensate. What I needed for that was basically a model of the <em>perceived</em> colors
of the LED tape's color channels. A way of doing his is to record the spectra of all color channels and then evaluate
their respective XYZ coordinates. If all three channels are measured in one go with the same setup the relative
magnitudes of the channels in XYZ will be accurate.</p>
<p>To map any color to the LEDs, the color's XYZ coordinates simply have to be mapped onto the linear coordinate system
produced by these three points within XYZ. LEDs are mostly linear in their luminous flux vs. current characteristic so
this model will be adequate. The spectral integrals mapping the channels' measured responses to XYZ need only be
calculated once and their results can be used as scaling factors thereafter.</p>
</div>
<div class="section" id="measuring-the-spectrum">
<h2>Measuring the spectrum</h2>
<p>In order to compensate for the cheap LED tape's non-ideal performance I had to measure the LED's red, green and blue
channels' spectra. The obvious thing would be to go out and buy a <a class="reference external" href="https://en.wikipedia.org/wiki/Ultraviolet%E2%80%93visible_spectroscopy">spectrograph</a>, or ask someone to borrow theirs. The
former is kind of expensive, and I did not want to wait two weeks for the thing to arrive. The latter I could probably
not do every time I got new LED tape. Thus the only choice was to build my own.</p>
<p>Luckily, building your own spectrometer is really easy. The first thing you need is something that splits incident light
into its constituent wavelengths. In professional devices this is called the <em>`monochromator`_</em>, since it allows extraction
of small color bands from the spectrum. The second thing is some sort of optics that project the incident light onto a
screen behind the monochromator. In professional devices lenses or curved mirrors are used. In a simple homebrew job a
pinhole as you would use in a <a class="reference external" href="https://en.wikipedia.org/wiki/Pinhole_camera">camera obscura</a> does a remarkably nice job.</p>
<p>For the monochromator component several things could be used. A prism would work, but I did not have any. The
alternative is a <a class="reference external" href="https://en.wikipedia.org/wiki/Diffraction_grating">diffraction grating</a>. Professional gratings are quite specialized pieces of equipment and thus
rather expensive. Luckily, there is a common household item that works almost as well: A regular CD or DVD. The
microscopic grooves that are used to record data in a CD or DVD work the same as the grooves in a professional
diffraction grating.</p>
</div>
<div class="section" id="household-spectra">
<h2>Household spectra</h2>
<p>From this starting point, a few seconds on my favorite search engine yielded an <a class="reference external" href="http://www.candac.ca/candacweb/sites/default/files/BuildaSpectroscope.pdf">article by two researchers from the
National Science Museum in Tokyo</a> providing a nice blueprint for a simple cardboard-and-DVD construction for use in
classrooms. I replicated their device using a DVD and it worked beautifully. Daylight and several types of small LEDs I
had around did show the expected spectra. Small red, yellow, green, and blue LEDs showed narrow spectra, daylight one
continuous broad one, and white LEDs a continuous broad one with a distinct bright spot in the blue part. The
single-color LED spectra are quite narrow since they are determined by the LED's semiconductor's band gap, which is
specific to the semiconductor used and is quite precise. White LEDs are in fact a blue LED chip covered with a so-called
<em>phosphor</em>. This phosphor is not elementary phosphorus but an anorganic compound that absorbs the LED chip's blue light
and re-emits a broader spectrum of more yellow-ish wavelengths instead. The final LED spectrum is a superposition of
both spectra, with some of the original blue light leaking through the phosphor mixing with the broadband yellow
spectrum of the phosphor.</p>
<div class="subfigure" data-pagefind-ignore>
<figure>
<img src="images/spectrograph_step1_parts.jpg">
<figcaption>The ingredients. The cup of coffee and Madoka Magica DVD set are essential to the eventual
function of the appartus.</figcaption>
</figure>
<figure>
<img src="images/spectrograph_step2.jpg">
<figcaption>Step 1: Cut to size and mark down all holes as described in <a
href="http://www.candac.ca/candacweb/sites/default/files/BuildaSpectroscope.pdf">the manual</a></figcaption>
</figure>
<figure>
<img src="images/spectrograph_step3.jpg">
<figcaption>Step 2: Cut out all holes</figcaption>
</figure>
<figure>
<img src="images/spectrograph_step4_complete.jpg">
<figcaption>The finished result with the back side showing. The viewing window is on the bottom of the other
side.</figcaption>
</figure>
</div><p>Now that I had a spectrograph, I needed a somewhat predictable way of measuring the spectrum it gave me.</p>
</div>
<div class="section" id="measuring-a-spectrum">
<h2>Measuring a spectrum</h2>
<p>Pointing a camera at the spectrograph would be the obvious thing to do. This produces pretty images but has one critical
flaw: I wanted to acquire quantitative measurements of brightness across the spectrum. Since I don't have a precise
technical datasheet specifying the spectral response of any of my cameras I can't compare the absolute brightness of
different colors on their pictures. Some other sensor was needed.</p>
<figure data-pagefind-ignore>
<img src="images/daylight_spectrum_dvd.jpg">
<figcaption>The daylight spectrum as seen using a DVD as a grating.
<a href="https://commons.wikimedia.org/wiki/File:SpectresSolaires-DVD.jpg">Picture</a> by
<a href="https://commons.wikimedia.org/wiki/User:Xofc">Xofc from Wikimedia Commons</a>,
<a href="https://creativecommons.org/licenses/by-sa/4.0/">CC-BY-SA 4.0</a>
</figcaption>
</figure><div class="section" id="measuring-light-intensity">
<h3>Measuring light intensity</h3>
<p>Looking around my lab, I found a bag of <a class="reference external" href="https://dammedia.osram.info/media/resource/hires/osram-dam-2495903/SFH%202701.pdf">SFH2701</a> visible-light photodiodes. Their
datasheet includes their spectral response so I can compensate for that, allowing precise-ish absolute intensity
measurements. Just like LEDs, photodiodes are extremely linear across several orders of magnitude. The datasheet of the
classic <a class="reference external" href="http://www.vishay.com/docs/81521/bpw34.pdf">BPW34</a> photodiode shows that this photodiode's light current is exactly proportional to illuminance over at
least three orders of magnitude. The <a class="reference external" href="https://dammedia.osram.info/media/resource/hires/osram-dam-2495903/SFH%202701.pdf">SFH2701</a> datasheet does not include a similar graph but its performance will be
similar. The <a class="reference external" href="https://dammedia.osram.info/media/resource/hires/osram-dam-2495903/SFH%202701.pdf">SFH2701</a> photodiodes I had at hand were perfect for the job compared to the vintage <a class="reference external" href="http://www.vishay.com/docs/81521/bpw34.pdf">BPW34</a> since their
active sensing area is really small (0.6mm by 0.6mm) compared to the BPW34 (a whopping 3mm by 3mm). If I were to use a
<a class="reference external" href="http://www.vishay.com/docs/81521/bpw34.pdf">BPW34</a> I would have to insert some small apterture in front of it so it does not catch too broad a part of the
spectrum at once. The <a class="reference external" href="https://dammedia.osram.info/media/resource/hires/osram-dam-2495903/SFH%202701.pdf">SFH2701</a> is small enough that if I just point it at the projected spectrum directly I will
already get only a small part of the spectrum inside its 0.6mm active area.</p>
<p>To convert the photodiode's tiny photocurrent into a measurable voltage I built another copy of the <a class="reference external" href="https://en.wikipedia.org/wiki/Transimpedance_amplifier">transimpedance
amplifier</a> circuit I already used in the <a class="reference external" href="http://jaseg.de/blog/multichannel-led-driver/">multichannel LED driver</a>. A <a class="reference external" href="https://en.wikipedia.org/wiki/Transimpedance_amplifier">transimpedance amplifier</a> is an
amplifiert that produces a large voltage from a small current. The weird name comes from the fact that it works kind of
like an amplified resistor (which can be generalized as an <em>impedance</em> electrically). Apply a current to a resistor and
you get a voltage. A transimpedance amplifiert does the same with the difference that its input always stays at 0V,
making it look like an ideal current sink to the connected current source.</p>
<p>Transimpedance amplifiers are common in optoelectronics to convert small photocurrents to voltages. In this instance I
built a very simple circuit with a dampened transimpedance amplifier stage followed by a simple RC filter for noise
rejection and a regular non-inverting amplifier using another op-amp from the same chip to further boost the filtered
transimpedance amplifier output. I put all the passives setting amplifier response (the gain-setting resistors and the
filter resistor and capacitors) on a small removable adapter so I could easily change them if necessary. I put a small
trimpot on the virtual ground both amplifers use as a reference so I could trim that if necessary.</p>
<figure data-pagefind-ignore>
<img src="images/preamp_schematic.jpg" alt="A drawing of the photodiode preamplifier's schematic">
<figcaption>The photodiode preamplifier schematic. Schematic drawn with an unlicensed copy of
DaveCAD.</figcaption>
</figure><p>Following are pictures of the preamplifier board. The connectors on the top-left side are two copies of the analog
signal for the ADC and a small panel meter. The SMA connector is used as the photodiode input since coax cables are
generally low-leakage and have built-in shielding. The circuit is powered via the micro-USB connector and the analog
ground bias voltage can be adjusted using the trimpot.</p>
<p>For easy replacement, all passives setting gain and frequency response are on a small, pluggable carrier PCB made from a
SMD-to-DIP adapter.</p>
<p>Flying-wire construction is just fine for this low-frequency circuit. In a high-speed photodiode preamp, the
transimpedance amplifier circuit would be highly sensitive to stray capacitance, but we're not aiming at high speed
here.</p>
<div class="subfigure" data-pagefind-ignore>
<figure>
<img src="images/preamp_front.jpg">
<figcaption>The front side of the preamplifier board.</figcaption>
</figure>
<figure>
<img src="images/preamp_back.jpg">
<figcaption>The wiring of the photodiode preamp.</figcaption>
</figure>
</div><p>Given a way to measure intensity what remains missing is a way to scan a single photodiode across the spectrum.</p>
</div>
<div class="section" id="scanning-the-projection">
<h3>Scanning the projection</h3>
<p>A cheap linear stage can be found in any old CD or DVD drive. These drives use a small linear stage based on a
stepper-driven screw to move the laser unit radially. Removing the laser unit and connecting a leftover stepper driver
module I was left with a small linear stage with about 45 steps per cm without microstepping enabled. The driver I used
was an <a class="reference external" href="https://www.pololu.com/file/0J450/A4988.pdf">A4988</a> module that required at least 8V motor drive voltage. I used a small micro USB-input boost converter
module to generate a stable 10V supply for the motor driver, with the USB's 5V rail used as a logic supply for the motor
driver.</p>
<p>The <a class="reference external" href="https://dammedia.osram.info/media/resource/hires/osram-dam-2495903/SFH%202701.pdf">SFH2701</a> can easily be mounted to the linear stage using a small SMD breakout board glued in place with thin wires
connecting it to the transimpedance amplifier. The DVD drive linear stage is not very strong so it is important that
this wire does not put too much strain on it.</p>
<p>Above the photodiode, I mounted a small piece of paper on the linear stage to be used as a projection screen to align
the linear stage in front of the spectrometer viewing window. A line on the screen paper points to the photodiode die in
parallel to the linear stage allowing precise alignment.</p>
<p>The whole unit with photodiode preamplifier, linear stage, photodiode and stepper motor driver finally looks like this:</p>
<figure data-pagefind-ignore>
<img src="images/electronics_whole.jpg" alt="The complete electronics setup of the spectrograph. In the back
there is the DVD drive stepper stage. In front of it, mounted on a piece of wood are a small USB-to-12V
switching-regulator module to power the stepper motor in the top left, below on the bottom left is the
photodiode preamp and on the right is a breadboard with the stepper driver module and lots of jumper wires
interconnecting everything. On the right of the breadboard, a buspirate is attached to interface everything to a
computer. On the bottom edge of the piece of wood, two LED panel meters are mounted for readout of the preamp
output and the stepper supply voltages.">
<figcaption>The complete electronics setup. The buspirate on the right interfaces to a computer and controls the
stepper driver and ADC'es the preamp output. The two panel meters show the preamp output and stepper voltage for
setup.</figcaption>
</figure><p>The projection of the spectrum can be adjusted by moving the light source relative to the entry slot and by moving
around the grating DVD.</p>
</div>
<div class="section" id="the-capture-process">
<h3>The capture process</h3>
<p>To capture a spectrum, first the light source has to be mounted near the spectrograph's entry slot. The LED tape I
tested I just taped face-down directly into it. Next, the grating DVD has to be adjusted to make sure the spectrum
covers a sensible part of the photodiode's path. Mostly, this boils down to adjusting the photodiode distance and height
to match the vertical extent and wiggling the grating DVD to adjust the projection's horizontal position.</p>
<p>After the optics are set-up, the photodiode preamplifier has to be adjusted. In my experiments, most LED tape at 5GΩ
required a high-ish amplification. The goal in this step is to maximize the peak response of the preamp to be just
shy of its VCC rail to make best use of its dynamic range. To adjust the pre-amp, I took several very coarsely-spaced
measurements to give me an estimate of the peak while I did not yet know its precise location.</p>
<p>Since stray daylight totally swamped out the weak projection of the LED's spectrum I shielded the entire setup with a
small box made of black cardboard and two black t-shirts on top. This shielding proved adequate for all my measurements
but I had to be careful not to accidentially move the DVD that was stuck into the spectrograph with the shielding
t-shirts.</p>
<p>For capturing a single spectrum I wrote a small python script that will automatically move the stepper in adjustable
intervals and take two measurements at each point, one with the LED tape off that can be used for offset calibration and
one with the LED tape on. All measurements are stored in a sqlite database that can then be accesssed from other
scripts.</p>
<p>I built a small script that shows the progress of the current run and an jupyter notebook for data analysis. The jupyter
notebook is capable of live-updating a graph with the in-progress spectrum's data. This was quite useful as a sanity
check for when I made some mistake easy to spot in the resulting data.</p>
<p>After one color channel is captured, the LED tape has to be manually set to the next color and the next measurement can
begin.</p>
<figure data-pagefind-ignore>
<img src="images/raw_plot_cheap_rgb.svg" alt="A plot with three wide peaks, two large peaks on both sides and
one smaller one in the middle. The middle one overlaps the two on the sides. The large ones are about 2.5V in
amplitude. Overall, the plot is about 300 stepper steps wide with each peak being around 130 steps wide.">
<figcaption>A plot of the raw preamp output voltage versus stepper position. From left to right, the three peaks
are blue, green and red. Step 0 corresponds to the bottommost stepper position and the shortest wavelength.
</figcaption>
</figure></div>
<div class="section" id="data-analysis">
<h3>Data analysis</h3>
<p>Data analysis consists of three major steps: Offset- and stray light removal, wavelength and amplitude calibration and
color space mapping.</p>
<div class="section" id="offset-removal">
<h4>Offset removal</h4>
<p>The first task is to remove the offset caused by dark current as well as stray light of the LED's bright primary
reflection on the DVD. The LED is very bright and only a small part of its light gets reflected by the grating towards
the photodiode screen. The remaining part of the light is reflected onto the table in front of the DVD spectrograph.
Though I covered all of this with black cardboard, some of that light ultimately gets reflected onto the photodiode.
This causes a large offset, in particular in the blue part of the spectrum since in this part the photodiode is closest
to the spectrograph's opening.</p>
<p>The composite offset can be approximated with a second-order polynomial that is fitted to all the data outside of the
main peak's area. Since at this point the wavelength of each data point is still unknown this is done with a rough first
estimate of the three colors' peaks' locations and widths.</p>
</div>
<div class="section" id="wavelength-and-amplitude-calibration">
<h4>Wavelength- and amplitude calibration</h4>
<p>The photodiode's response is strongly wavelength-dependent. In particular in the blue band, the photodiode's sensitivity
gets very poor down to about 20% at the edge to ultraviolet. This effect is strong enough to move the apparent location
of the blue peak towards red.</p>
<figure data-pagefind-ignore>
<img src="images/photodiode_sensitivity.svg" alt="A plot of photodiode sensitivity against wavelength relative
to peak sensitivity at 820nm. The sensitivity rises from 20% at 380nm approximately linearly to 80% at 620nm,
then the rise rolls off.">
<figcaption>A plot of the photodiode's relative sensitivity in the visible spectrum. The sensitivity is
normalized against its peak at 820nm.
</figcaption>
</figure><p>The problem is that in order to remove this non-linearity, we would already have to know the wavelength of the measured
light. Since I don't, I settled for a two-step process. First, a coarse wavelength calibration is done relative to the
red peak and the short-wavelength edge of the blue peak. The photodiode measurements are then sensitivity-corrected
using this coarse measurement. Then all three channel peaks are measured in the resulting data and a fine wavelength
estimate is produced by a least-squares fit of a linear function. This fine estimate is then used for a second
sensitivity correction of all original measurements and the scale is changed from stepper motor step count to
wavelength in nanometers.</p>
<figure data-pagefind-ignore>
<img src="images/processed_plot_cheap_rgb.svg" alt="A plot with three wide peaks, all three of different
heights. The leftmost peak is highest at 6nA, the middle peak lowest at 1.6nA and the rightmost peak in between
at 4nA. The middle one overlaps the two on the sides. Overall, the plot spans about 300nm on its x axis with
each peak being around 100nm wide.">
<figcaption>A plot of the processed measurements. From left to right, the three peaks are blue, green and red.
</figcaption>
</figure><!-- FIXME re-do these measurements, avoiding clipping -->
<!-- FIXME re-do calibration using CCFL -->
<!-- FIXME calibration for brightness imbalance due to wedge-shaped projection of spectrum -->
</div>
<div class="section" id="color-space-mapping">
<h4>Color space mapping</h4>
<p>Finally, to achieve the objective of measuring the LED tape's channels' precise color coordinates the measured spetra
have to be matched against the color spaces' <em>color matching functions</em>. The color matching functions describe how
strong the color space's idealized <em>standard observer</em> would react to light at a particular wavelength. Going from a
measured spectrum to color coordinates XYZ works by integrating over the product of the measurement and each color
coordinate's color matching function.</p>
<p>The result are three color coordinates X, Y and Z for each channel R, G and B yielding nine coordinates in total. When
written as a matrix conversion between XYZ color space and LED-RGB color space is as simple as multiplying that matrix
(or its inverse) and a vector from one of the color spaces.</p>
<p>In XYZ space, the set of colors that can be produced with this LED tape is described by the <a class="reference external" href="https://en.wikipedia.org/wiki/Parallelepiped">parallelepiped</a> spanned by
the three channel's XYZ vectors. In the following figures, you can see a three-dimensional model of the RGB LED's color
space (colorful) as well as sRGB (white) for comparison plotted within CIE 1931 XYZ. There is no natural map to scale
both so for this illustration the LED color space has been scaled to fit. These figures were made with blender and a few
lines of python. The blender project file including all settings and the python script to generate the color space
models can be found in the <a class="reference external" href="https://github.com/jaseg/led_drv">project repo</a>.</p>
<figure data-pagefind-ignore>
<video controls loop>
<source src="video/led_within_srgb_scale=1.0.mkv" type="video/h264">
<source src="video/led_within_srgb_scale=1.0.webm" type="video/webm">
Your browser does not support the HTML5 video tag.
</video>
<figcaption>Illustration of the measured LED color space scaled to fit within XYZ with sRGB (light gray) for
comparison. The thick, white line is the spectral locus.
<a href="video/led_within_srgb_scale=1.0.mkv">mkv/h264 download</a> /
<a href="video/led_within_srgb_scale=1.0.webm">webm download</a>
</figcaption>
</figure><p>As you can see, the result is pretty disappointing. The LED's color space parallepiped is very narrow, which is because
the blue channel is much brighter than the other two channels. An easy fix for this is to scale-up the RGB space and
drop any values outside XYZ. The scaling factor is a trade-off between color space coverage and brightness. You can
produce the most colors when you clip all channels to brightness of the weakest channel (green in this case), but that
will make the result very dim. Scaling brightness like that stretches the RGB parallelepiped along its major axis. Up to
a point the number of possible colors (the gamut) increases at expense of maximum brightness. When the parallelepiped is
stretched far enought for all three channel vectors to be outside the 1,1,1 XYZ-cube, maximum brightness continues to
decrease but the gamut stays constant. I don't know a simple scientific way to solve this problem, so I just played
around with a couple of factors and settled on 2.5 as a reasonable compromise. Below is an illustration.</p>
<figure>
<video controls loop>
<source src="video/led_within_srgb_fancy_camera_path_scale=2.5.mkv" type="video/h264">
<source src="video/led_within_srgb_fancy_camera_path_scale=2.5.webm" type="video/webm">
Your browser does not support the HTML5 video tag.
</video>
<figcaption>Illustration of the measured LED color space at scale factor 2.5 within XYZ with sRGB (light gray)
for comparison. The thick, white line is the spectral locus.
<a href="video/led_within_srgb_fancy_camera_path_scale=2.5.mkv">mkv/h264 download</a> /
<a href="video/led_within_srgb_fancy_camera_path_scale=2.5.webm">webm download</a>
</figcaption>
</figure></div>
</div>
</div>
<div class="section" id="firmware-implementation">
<h2>Firmware implementation</h2>
<p>In the end, the above measurements yield two matrices: One for mapping XYZ to RGB, and one for mapping RGB to XYZ. Of
the several versions of CIE XYZ I chose the CIE 1931 XYZ color space as a basis for the firmware because it is most
popular. Mapping a color coordinate in one color space to the other is as simple as performing nine floating-point
multiplications and six additions. Mapping Lab or Lch to RGB is done by first mapping Lab/Lch to XYZ, then XYZ to RGB.
Lab to XYZ is somewhat complex since it requires a floating-point power for gamma correction, but any self-respecting
libc will have one of those so this is still no problem. Lch also requires floating-point sine and cosine functions, but
these should still be no problem on most hardware.</p>
<p>My implementation of these conversions in the ESP8266 firmware of my <a class="reference external" href="http://jaseg.de/blog/wifi-led-driver/">Wifi LED driver</a> can be found <a class="reference external" href="https://github.com/jaseg/esp_led_drv/blob/master/user/led_controller.c">on Github</a>. You
can view the Jupyter notebook most of the analysis above <a class="reference external" href="http://nbviewer.jupyter.org/github/jaseg/led_drv/blob/master/doc/Spectrum%20Measurement.ipynb">here</a>.</p>
</div>
</div>
</main><footer>
Copyright © 2025 Jan Sebastian Götte
/ <a href="/about/">About</a>
/ <a href="/imprint/">Imprint</a>
</footer>
<script type="text/javascript" src="/pagefind/pagefind-ui.js" defer></script>
<script>
window.addEventListener('DOMContentLoaded', (event) => {
new PagefindUI({element: "#search", showSubResults: true});
});
</script>
<script type="speculationrules">
{
"prerender": [
{
"source": "document",
"where": {
"and": [
{"href_matches": "/*"}
]
},
"eagerness": "moderate"
}
]
}
</script>
</body>
</html>

0
content/blog/led-characterization/video/led_within_srgb_fancy_camera_path_scale=2.5.mkv → blog/led-characterization/video/led_within_srgb_fancy_camera_path_scale=2.5.mkv
View file

0
content/blog/led-characterization/video/led_within_srgb_fancy_camera_path_scale=2.5.webm → blog/led-characterization/video/led_within_srgb_fancy_camera_path_scale=2.5.webm
View file

0
content/blog/led-characterization/video/led_within_srgb_scale=1.0.mkv → blog/led-characterization/video/led_within_srgb_scale=1.0.mkv
View file

0
content/blog/led-characterization/video/led_within_srgb_scale=1.0.webm → blog/led-characterization/video/led_within_srgb_scale=1.0.webm
View file

0
content/blog/led-characterization/video/led_within_srgb_scale=2.5.mkv → blog/led-characterization/video/led_within_srgb_scale=2.5.mkv
View file

0
content/blog/led-characterization/video/led_within_srgb_scale=2.5.webm → blog/led-characterization/video/led_within_srgb_scale=2.5.webm
View file

0
content/blog/led-characterization/video/led_within_srgb_scale=3.mkv → blog/led-characterization/video/led_within_srgb_scale=3.mkv
View file

0
content/blog/led-characterization/video/led_within_srgb_scale=3.webm → blog/led-characterization/video/led_within_srgb_scale=3.webm
View file

0
content/blog/led-characterization/video/sRGB.mkv → blog/led-characterization/video/sRGB.mkv
View file

0
content/blog/led-characterization/video/sRGB.webm → blog/led-characterization/video/sRGB.webm
View file

0
content/blog/led-characterization/video/scale=1.mkv → blog/led-characterization/video/scale=1.mkv
View file

0
content/blog/led-characterization/video/scale=1.webm → blog/led-characterization/video/scale=1.webm
View file

0
content/blog/led-characterization/video/scale=2.5.mkv → blog/led-characterization/video/scale=2.5.mkv
View file

0
content/blog/led-characterization/video/scale=2.5.webm → blog/led-characterization/video/scale=2.5.webm
View file

0
content/blog/led-characterization/video/scale=5.mkv → blog/led-characterization/video/scale=5.mkv
View file

0
content/blog/led-characterization/video/scale=5.webm → blog/led-characterization/video/scale=5.webm
View file

0
content/blog/multichannel-led-driver/images/asymmetric_iled.svg → blog/multichannel-led-driver/images/asymmetric_iled.svg
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 69 KiB

After

Width:  |  Height:  |  Size: 69 KiB

Before After
Before After

0
content/blog/multichannel-led-driver/images/asymmetric_vgate.svg → blog/multichannel-led-driver/images/asymmetric_vgate.svg
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 58 KiB

After

Width:  |  Height:  |  Size: 58 KiB

Before After
Before After

0
content/blog/multichannel-led-driver/images/bcm_schema.jpg → blog/multichannel-led-driver/images/bcm_schema.jpg
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 1.6 MiB

After

Width:  |  Height:  |  Size: 1.6 MiB

Before After
Before After

0
content/blog/multichannel-led-driver/images/corrected_brightness_sim.svg → blog/multichannel-led-driver/images/corrected_brightness_sim.svg
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 22 KiB

After

Width:  |  Height:  |  Size: 22 KiB

Before After
Before After

0
content/blog/multichannel-led-driver/images/driver_linearity_raw.svg → blog/multichannel-led-driver/images/driver_linearity_raw.svg
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 29 KiB

After

Width:  |  Height:  |  Size: 29 KiB

Before After
Before After

0
content/blog/multichannel-led-driver/images/driver_output_ltspice_schematic.jpg → blog/multichannel-led-driver/images/driver_output_ltspice_schematic.jpg
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 856 KiB

After

Width:  |  Height:  |  Size: 856 KiB

Before After
Before After

0
content/blog/multichannel-led-driver/images/driver_pcb_built.jpg → blog/multichannel-led-driver/images/driver_pcb_built.jpg
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 1.2 MiB

After

Width:  |  Height:  |  Size: 1.2 MiB

Before After
Before After

0
content/blog/multichannel-led-driver/images/driver_ringing_strong.jpg → blog/multichannel-led-driver/images/driver_ringing_strong.jpg
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 279 KiB

After

Width:  |  Height:  |  Size: 279 KiB

Before After
Before After

0
content/blog/multichannel-led-driver/images/driver_ringing_weak.jpg → blog/multichannel-led-driver/images/driver_ringing_weak.jpg
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 285 KiB

After

Width:  |  Height:  |  Size: 285 KiB

Before After
Before After

0
content/blog/multichannel-led-driver/images/led_strip_alight.jpg → blog/multichannel-led-driver/images/led_strip_alight.jpg
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 1.7 MiB

After

Width:  |  Height:  |  Size: 1.7 MiB

Before After
Before After

0
content/blog/multichannel-led-driver/images/linearization_setup.jpg → blog/multichannel-led-driver/images/linearization_setup.jpg
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 1.9 MiB

After

Width:  |  Height:  |  Size: 1.9 MiB

Before After
Before After

0
content/blog/multichannel-led-driver/images/olsndot_output_schematic.jpg → blog/multichannel-led-driver/images/olsndot_output_schematic.jpg
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 1.3 MiB

After

Width:  |  Height:  |  Size: 1.3 MiB

Before After
Before After

0
content/blog/multichannel-led-driver/images/olsndot_pcb.png → blog/multichannel-led-driver/images/olsndot_pcb.png
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 134 KiB

After

Width:  |  Height:  |  Size: 134 KiB

Before After
Before After

0
content/blog/multichannel-led-driver/images/olsndot_schematic.png → blog/multichannel-led-driver/images/olsndot_schematic.png
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 354 KiB

After

Width:  |  Height:  |  Size: 354 KiB

Before After
Before After

0
content/blog/multichannel-led-driver/images/overshoot_sim_r0.svg → blog/multichannel-led-driver/images/overshoot_sim_r0.svg
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 58 KiB

After

Width:  |  Height:  |  Size: 58 KiB

Before After
Before After

0
content/blog/multichannel-led-driver/images/overshoot_sim_r100.svg → blog/multichannel-led-driver/images/overshoot_sim_r100.svg
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 56 KiB

After

Width:  |  Height:  |  Size: 56 KiB

Before After
Before After

0
content/blog/multichannel-led-driver/images/pwm_schema.jpg → blog/multichannel-led-driver/images/pwm_schema.jpg
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 820 KiB

After

Width:  |  Height:  |  Size: 820 KiB

Before After
Before After

0
content/blog/multichannel-led-driver/images/uncorrected_brightness_sim.svg → blog/multichannel-led-driver/images/uncorrected_brightness_sim.svg
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 20 KiB

After

Width:  |  Height:  |  Size: 20 KiB

Before After
Before After

405
blog/multichannel-led-driver/index.html Normal file
View file

@ -0,0 +1,405 @@
<!DOCTYPE html>
<html><head>
<meta charset="utf-8">
<title>32-Channel LED tape driver | Home</title>
<meta name="description" content="">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="mobile-web-app-capable" content="yes">
<meta name="color-scheme" content="dark light">
<link rel="stylesheet" href="/style.css">
<link rel="preload" href="/fonts/roboto_slab/RobotoSlab-VariableFont_wght.ttf" as="font" type="font/ttf" crossorigin />
<link rel="preload" href="/fonts/nyght-serif-main/fonts/WEB/NyghtSerif-Regular.woff2" as="font" type="font/woff2" crossorigin />
<link rel="preload" href="/fonts/nyght-serif-main/fonts/WEB/NyghtSerif-Bold.woff2" as="font" type="font/woff2" crossorigin />
<link rel="preload" href="/fonts/nyght-serif-main/fonts/WEB/NyghtSerif-BoldItalic.woff2" as="font" type="font/woff2" crossorigin />
</head>
<body><nav>
<div class="internal">
<a href="/" title="Home">Home</a>
<a href="/blog/" title="Blog">Blog</a>
<a href="/projects/" title="Projects">Projects</a>
<a href="/about/" title="About">About</a>
</div>
<div class="search">
<div id="search"></div>
</div>
<div class="external">
<a href="https://git.jaseg.de/" title="cgit">cgit</a>
<a href="https://github.com/jaseg" title="Github">Github</a>
<a href="https://gitlab.com/neinseg" title="Gitlab">Gitlab</a>
<a href="https://chaos.social/@jaseg" title="Mastodon">Mastodon</a>
</span>
</nav>
<header>
<h1>32-Channel LED tape driver</h1>
<ul class="breadcrumbs">
<li><a href="/">jaseg.de</a></li>
<li><a href="/blog/">Blog</a></li><li><a href="/blog/multichannel-led-driver/">32-Channel LED tape driver</a></li>
</ul>
<strong>2018-05-02</strong>
</header>
<main data-pagefind-body>
<div class="document">
<div class="section" id="theoretical-basics">
<h2>Theoretical basics</h2>
<p>Together, a friend and I outfitted the small staircase at Berlin's Chaos Computer Club with nice, shiny RGB-WW LED tape
for ambient lighting. This tape is like regular RGB tape but with an additional warm white channel, which makes for much
more natural pastels and whites. There are several variants of RGBW tape. Cheap ones have separate RGB and white LEDs,
which is fine for indirect lighting but does not work for direct lighting. Since we wanted to mount our tape in channels
at the front of the steps, we had to use the slightly more expensive variant with integrated RGBW LEDs. These are LEDs
in the 5050 (5.0mm by 5.0mm) form factor common with RGB LEDs that have a small section divided off for the white
channel. The red, green and blue LED chips sit together in the larger section covered with clear epoxy and the white
channel is made up from the usual blue LED inside a yellow phosphor in the smaller section.</p>
<p>Since we wanted to light up all of 15 steps, and for greatest visual effect we would have liked to be able to control
each step individually we had to find a way to control 60 channels of LED tape with a reasonable amount of hardware.</p>
<p>LED tape has integrated series resistors and runs off a fixed 12V or 24V constant-voltage supply. This means you don't
need a complex constant-current driver as you'd need with high-power LEDs. You can just hook up a section of LED tape
to a beefy MOSFET to control it. Traditionally, you would do <em>Pulse Width Modulation</em> (PWM) on the MOSFET's input to
control the LED tape's brightness.</p>
<div class="section" id="pulse-width-modulation">
<h3>Pulse Width Modulation</h3>
<p><a class="reference external" href="https://en.wikipedia.org/wiki/Pulse-width_modulation">Pulse Width Modulation</a> is a technique of controlling the brightness of a load such as an LED with a digital signal.
The basic idea is that if you turn the LED on and off much too fast for anyone to notice, you can control its power by
changing how long you turn it on versus how long you leave it off.</p>
<p>PWM divides each second into a large number of periods. At the beginning of each period, you turn the LED on. After
that, you wait a certain time until you turn it off. Then, you wait for the next period to begin. The periods are always
the same length but you can set when you turn off the LED. If you turn it off right away, it's off almost all the time
and it looks like it's off to your eye. If you turn it off right at the end, it's on almost all the time and it looks
super bright to your eye. Now, if you turn it off halfway into the cycle, it's on half the time and it will look to your
eye as half as bright as before. This means that you can control the LED's brightness with only a digital signal and
good timing.</p>
<figure data-pagefind-ignore>
<img src="images/pwm_schema.jpg" alt="A visualization of PWM at different duty cycles.">
<figcaption>Waveforms of two PWM cycles at different duty cycles.</figcaption>
</figure><p>PWM works great if you have a dedicated PWM output on your microcontroller. It's extremely simple in both hardware and
software. Unfortunately for us, controlling 32 channels with PWM is not that easy. Cheap microcontrollers only have <a class="reference external" href="https://www.nxp.com/parametricSearch#/&amp;c=c731_c380_c173_c161_c163&amp;page=1">a
handful of hardware PWM outputs</a>, so we'd either have to do everything in software, bit-banging our LED modulation, or
we'd have to use a dedicated chip.</p>
<p>Doing PWM in software is both error-prone and slow. Since the maximum dynamic range of a PWM signal is limited by the
shortest duty cycle it can do, software PWM being slow means it has poor PWM resolution at maybe 8 bits at most. Poor
color resolution is not a problem if all you're doing is to fade around the <a class="reference external" href="https://en.wikipedia.org/wiki/HSL_and_HSV">HSV rainbow</a>, but for ambient lighting
where you <em>really</em> want to control the brightness down to a faint shimmer you need all the color resolution you can get.</p>
<p>If you rule out software PWM, what remains are dedicated <a class="reference external" href="http://www.ti.com/lit/ds/symlink/tlc5940.pdf">hardware PWM controllers</a>. Most of these have either of three
issues:</p>
<ul class="simple">
<li>They're expensive</li>
<li>They don't have generous PWM resolution either (12 bits if you're lucky)</li>
<li>They're meant to drive small LEDs such as a 7-segment display directly and you can't just hook up a MOSFET to their
output</li>
</ul>
<p>This means we're stuck in a dilemma between two poor solutions if we'd want to do PWM. Luckily for us, PWM is not the
only modulation in town.</p>
</div>
<div class="section" id="binary-code-modulation">
<h3>Binary Code Modulation</h3>
<p>PWM is the bread-and-butter of the maker crowd. Everyone and their cat is doing it and it works really well most of the
time. Unbeknownst to most of the maker crowd, there is however another popular modulation method that's mostly used in
professional LED systems: Enter <a class="reference external" href="http://www.batsocks.co.uk/readme/art_bcm_1.htm">*Binary Code Modulation* (BCM)</a>.</p>
<p>BCM is to PWM sort of what barcodes are to handwriting. While PWM is easy to understand and simple to implement if all
you have is a counter and an IO pin, BCM is more complicated. On the other hand, computers can do complicated and BCM
really shines in multi-channel applications.</p>
<p>Similar to PWM, BCM works by turning on and off the LED in short periods fast enough to make your eye perceive it as
partially on all the time. In PWM the channel's brightness is linearly dependent on its duty cycle, i.e. the percentage
it is turned on. In PWM the duty cycle D is the total period T divided by the on period T_on. The issue with doing PWM
on many channels at once is that you have to turn off each channel at the exact time to match its duty cycle.
Controlling many IO pins at once with precise timing is really hard to do in software.</p>
<p>BCM avoids this by further dividing each period into smaller periods which we'll call <em>bit periods</em> and splitting each
channel's duty cycle into chunks the size of these bit periods. The amazingly elegant thing in BCM now is that as you
can guess from the name these bit periods are weighted in powers of two. Say the shortest bit period lasts 1
microsecond. Then the second-shortest bit period is 2 microseconds and the third is 4, the fifth 8, the sixth 16 and so
on.</p>
<figure data-pagefind-ignore>
<img src="images/bcm_schema.jpg" alt="A visualization of BCM at different duty cycles.">
<figcaption>Waveforms of a single 4-bit BCM cycle at different duty cycles. This BCM can produce 16 different
levels.</figcaption>
</figure><p>Staggered like this, you turn on the LED for integer value of microseconds by turning it on in the bit periods
corresponding to the binary bits of that value. If I want my LED to light for 19 microseconds every period, I turn it on
in the 16 microsecond bit period, the 2 microsecond bit period and the 1 microsecond bit period and leave it off for the
4 and 8 mircosecond bit periods.</p>
<p>Now, how this is better instead of just more complicated than plain old PWM might not be clear yet. But consider this:
Turning on and off a large number of channels, each at its own arbitrary time is hard because doing the timing in
software is hard. We can't use hardware timers since we only have two or three of those, and we have 32 channels.
However, we can use one hardware timer to trigger a really cheap external latch to turn on or off the 32 channels all at
once. With this setup, we can only controll all channels at once, but we can do so with very precise timing.</p>
<p>All we need to do is to set our timer to the durations of the BCM bit periods, and we can get the same result as we'd
get with PWM with only one hardware timer and a bit of code that is not timing-critical anymore.</p>
<div class="section" id="applications-of-binary-code-modulation">
<h4>Applications of Binary Code Modulation</h4>
<p>BCM is a truly wondrous technique, and outside of hobbyist circles it is in fact very widely known. Though we're using
it to control just 32 channels here, you can do much more channels without any problems. The most common application
where BCM is invariably used is <em>any</em> kind of LED screen. Controlling the thousands and thousands of LEDs in an LED
screen with PWM with a dedicated timer for each LED would not be feasible. With BCM, all you need to dedicate to a
single LED is a flipflop (or part of one if you're multiplexing). In fact, there is a whole range of <a class="reference external" href="http://www.vabolis.lt/stuff/MBI5026.pdf">ICs with no other
purpose than to enable BCM on large LED matrices</a>. Basically, these are a
high-speed shift register with latched outputs much like the venerable <a class="reference external" href="http://www.ti.com/lit/ds/symlink/sn74hc595.pdf">74HC595</a>, only their outputs are constant-current
sinks made so that you can directly connect an LED to them.</p>
</div>
<div class="section" id="running-bcm-on-led-tape">
<h4>Running BCM on LED tape</h4>
<p>In our case, we don't need any special driver chips to control our LED tape. We just connect the outputs of a <a class="reference external" href="http://www.ti.com/lit/ds/symlink/sn74hc595.pdf">74HC595</a>
shift register to one <a class="reference external" href="https://en.wikipedia.org/wiki/MOSFET">MOSFET</a> each, and then we directly connect the LED tape to these MOSFETs. The MOSFETs allow us to
drive a couple of amps into the LED tape from the weak outputs of the shift register.</p>
<p>The BCM timing is done by hooking up two timer channels of our microcontroller to the shift registers <em>strobe</em> and
<em>reset</em> inputs. We set the timer to PWM mode so we can generate pulses with precise timing. At the beginning of each
bit period, a pulse will strobe the data for this bit period that we shifted in previously. At the end of the bit
period, one pulse will reset the shift register and one will strobe the freshly-reset zeros into the outputs.</p>
<figure data-pagefind-ignore>
<img src="images/olsndot_output_schematic.jpg" alt="From left to right, we see the STM32, one of the shift
registers, and the LEDs and MOSFETs. The LED tape is driven to ground by the MOSFETs, which are in turn directly
driven from the shift register outputs. The shift register is wired up to the STM32 with its clock and data
inputs on SCK and MOSI and its RESET and STROBE inputs on channel 2 and 3 of timer 1.">
<figcaption>
The schematic of a single output of this LED driver. Multiple shift register stages can be cascaded.
</figcaption>
</figure><p>Our implementation of this system runs on an <a class="reference external" href="http://www.st.com/resource/en/datasheet/stm32f030f4.pdf">STM32F030F4P6</a>, the smallest, cheapest ARM microcontroller you can get from
ST. This microcontroller has only 16kB of flash and 1kB of RAM, but that's plenty for our use. We use its SPI controller
to feed the modulation data to the shift registers really fast, and we use two timer channels to control the shift
registers' reset and strobe.</p>
<p>We can easily cascade shift registers without any ill side-effects, and even hundreds of channels should be no problem
for this setup. The only reason we chose to stick to a 32-channel board is the mechanics of it. We thought it would be
easier to have several small boards instead of having one huge board with loads of connectors and cables coming off it.</p>
<p>The BOM cost per channel for our system is 3ct for a reasonable MOSFET, about 1ct for one eighth of a shift register
plus less than a cent for one resistor between shift register and MOSFET. In the end, the connectors are more expensive
than the driving circuitry.</p>
</div>
</div>
</div>
<div class="section" id="hardware-design">
<h2>Hardware design</h2>
<p>From this starting point, we made a very prototype-y hardware design for a 32-channel 12V LED tape driver. The design is
based on the <a class="reference external" href="http://www.st.com/resource/en/datasheet/stm32f030f4.pdf">STM32F030F4P6</a> driving the shift registers as explained above. The system is controlled through an <a class="reference external" href="https://en.wikipedia.org/wiki/RS-485">RS485</a>
bus that is connected up to the microcontroller's UART using an <a class="reference external" href="https://datasheets.maximintegrated.com/en/ds/MAX1487-MAX491.pdf">MAX485</a>-compatible RS485 transceiver. The LED tape is
connected using 9-pin <a class="reference external" href="https://en.wikipedia.org/wiki/D-subminiature">SUB-D</a> connectors since they are cheap and good enough for the small current of our short segments
of LED tape. The MOSFETs we use are small <a class="reference external" href="http://www.nxp.com/documents/outline_drawing/SOT23.pdf">SOT-23</a> logic-level MOSFETs. In various prototypes we used both International
Rectifier's <a class="reference external" href="https://www.infineon.com/dgdl/?fileId=5546d462533600a4015356686fed261f">IRLML6244</a> as well as Alpha &amp; Omega Semiconductor's <a class="reference external" href="http://aosmd.com/pdfs/datasheet/AO3400.pdf">AO3400</a>. Both are good up to about 30V/5A. Since we're
only driving about 2m of LED tape per channel we're not going above about 0.5A and the MOSFETs don't even get warm.</p>
<div class="section" id="switching-nonlinearities">
<h3>Switching nonlinearities</h3>
<p>During testing of our initial prototype, we noticed that the brightness seemed to jump around when fading to very low
values. It turned out that our extremely simple LED driving circuit consisting of only the shift register directly
driving a MOSFET, which in turn directly drives the LED tape was maybe a little bit too simple. After some measurements
it turned out that we were looking at about 6Vpp of ringing on the driver's output voltage. The picture below is the
voltrage we saw on our oscilloscope on the LED tape.</p>
<figure data-pagefind-ignore>
<img src="images/driver_ringing_strong.jpg" alt="Strong ringing on the LED voltage waveform edge at about
100% overshoot during about 70% of the cycle time.">
<figcaption>Bad ringing on the LED output voltage caused by wiring inductance. Note that the effect on the
actual LED current is less bad than this looks since the LED's V/I curve is nonlinear.</figcaption>
</figure><div class="section" id="dynamic-switching-behavior-cause-and-effect">
<h4>Dynamic switching behavior: Cause and Effect</h4>
<p>A bit of <a class="reference external" href="http://www.analog.com/en/design-center/design-tools-and-calculators/ltspice-simulator.html">LTSpice</a> action later we found that the inductance of the few metres of cable leading to the LED tape is the
likely culprit. The figure below is the schematic used for the simulations.</p>
<figure data-pagefind-ignore>
<img src="images/driver_output_ltspice_schematic.jpg" alt="The LTSpice schematic of one output of the driver,
taking into account the shift register's output ESR and the wiring ESL.">
<figcaption>The schematic of the simulation in LTSpice</figcaption>
</figure><p>As tested, the driver does not include any per-output smoothing so the ~.5A transient on each BCM cycle hits the cable
in full. Combined with the cable inductance, this works out to a considerable lag of the rising edge of the LED
current, and bad ringing on its falling edge. Below is the voltage on the LED output from an LTSpice simulation of our
driver.</p>
<figure data-pagefind-ignore>
<img src="images/overshoot_sim_r0.svg" alt="The result of the LTSpice simulation of our driver output. The LED
current shows similar ringing to what we measured using the oscilloscope. Interestingly, the gate voltage shows
strong ringing, too.">
<figcaption>The result of our LTSpice simulation. This simulation assumes 1µH of wiring inductance and 50Ω of
output impedance on the part of the shift register. The ringing at the gate visible in the gate voltage graph is
due to feed-through of the ringing at the output through the MOSFET's parasitic Cgd.</figcaption>
</figure><p>We were able to reduce the rining and limit the effect somewhat by putting a 220Ω series resistor in between the shift
register output and the MOSFET gate. This resistor forms an RC circuit with the MOSFET's nanofarad or two of gate
capacitance. The result of this is that the LED current passing the wire's ESL rises slightly more slowly and thus the
series inductance gets excited slightly less, and the overshoot decreases. Below is a picture of the waveform with the
damping resistor in place and a picture of our measurement for comparison. The resistor values don't agree perfectly
since the estimated ESL and stray capacitance of the wiring is probably way off.</p>
<figure data-pagefind-ignore>
<img src="images/driver_ringing_weak.jpg" alt="Weak ringing on the LED voltage waveform edge at about 30%
overshoot during about 20% of the cycle time.">
<figcaption>Adding a resistor in front of the MOSFET gate to slow the transition damped the ringing somewhat,
but ultimately it cannot be eliminated entirely. Note how you can actually see the miller plateau on the
trailing edge of this signal.
</figcaption>
</figure><figure data-pagefind-ignore>
<img src="images/overshoot_sim_r100.svg" alt="The result of the LTSpice simulation of our driver output with an
extra 100 Ohms between shift register output and MOSFET gate. Similar to the oscilloscope measurement the
ringing is much reduced in its amplitude.">
<figcaption>The LTSpice simulation result with the same parameters as above but with an extra 100Ω between the
shfit register's output and the MOSFET's gate.</figcaption>
</figure><p>A side effect of this fix is that now the effective on-time of the LED tape is much longer than the duty cycle at the
shift register's output at very small duty cycles (1µs or less). This is caused by the MOSFET's <a class="reference external" href="https://www.vishay.com/docs/68214/turnonprocess.pdf">miller
plateau</a>. For illustration, below is a graph of both the excitation waveform (the boxy line) and the resulting LED
current (the other ones) both without damping (top) and with 220Ω damping (bottom). As you can see the effective duty
cycle of the LED current is not at all equal to the 50% duty cycle of the excitation square wave.</p>
<figure data-pagefind-ignore>
<img src="images/asymmetric_iled.svg" alt="The result of an LTSpice simulation of the LED duty cycle without and
with damping. Dampening widens the LED current waveform from 50% duty cycle with sharp edges to about 80% duty
cycle with soft edges.">
<figcaption>Simulated LED duty cycle with and without damping. The damping resistance used in this simulation
was 220Ω.</figcaption>
</figure><figure data-pagefind-ignore>
<img src="images/asymmetric_vgate.svg" alt="The gate voltages in the spice simulation above. The undamped
response shows sharp edges with the miller plateau being a barely noticeable step, but with strong ringing on
the trailing edge. The damped response shows RC-like slow-edges, but has wide miller plateaus on both edges
adding up to about 50% of the pulse width.">
<figcaption>The MOSFET gate voltage from the simulation in the figure above. You can clearly see how the miller
plateau (the horizontal part of the trace at about 1V) is getting much wider with added damped, and how the
resulting gate charge/discharge curve is not at all that of a capacitor anymore.</figcaption>
</figure><p>In conclusion, we have three major causes for our calculated LED brightness not matching reality:</p>
<ul class="simple">
<li>Ringing of the equivalent series inductance of the wiring leading up to the LED tape</li>
<li>Miller plateau lag</li>
<li>The damping resistor and the MOSFET gate forming an RC filter that helps with wire ESL ringing but worsens the miller
plateau issue and deforms the LED current edges.</li>
</ul>
<p>Added up, these three effects yield a picture that agrees well with our simulations and measurements. The overall effect
is neglegible at long period durations (&gt;10µs), but gets really bad at short period durations (&lt;1µs). The effect is
non-linear, so correcting for it is not as simple as adding an offset.</p>
</div>
<div class="section" id="measuring-led-tape-brightness">
<h4>Measuring LED tape brightness</h4>
<p>In order to correct for the nonlinearities mentioned above, we decided to implement a lookup table mapping BCM period to
actual timer setting. That is, each row of the table contains the actual period length we need to set the
microcontroller's timer to in order to get our intended brightness steps.</p>
<p>To calibrate our driver, we needed a setup for reproducible measurement of the relative brightness of our LED tape at
different settings. Absolute brightness is not of interest to us as the eye can't perceive it. To perform the
calibration, the LED driver is set to enable each single BCM period in turn, i.e. brightness values 1, 2, 4, 8, 16 etc.</p>
<p>The setup we used to measure the LED tape's brightness consists of a bunch of LED tape stuck into a tin can for
shielding against both stray light and electromagnetic interference and a photodiode looking at the LED tape. We used
the venerable <a class="reference external" href="http://www.vishay.com/docs/81521/bpw34.pdf">BPW34</a> photodiode in our setup as I had a bunch leftover from another project and because they are quite
sensitive owing to their physically large die area.</p>
<figure data-pagefind-ignore>
<img src="images/linearization_setup.jpg" alt="The led measurement setup consists of several PCBs and a
breadboard linked with a bunch of wires and a big tin can to shield the LEDs and the photodiode. A large sub-D
connector is put into the top of the tin can as a feed-through for the LED tape's control signals and the
photodiode signal. In the background the control laptop is visible.">
<figcaption>The LED brighness measurement setup. The big tin can contains a bunch of LED tape and the
photodiode. The breadboard on the right is used for the photodiode preamplifier and for jumpering around the LED
tape's channels. The red board next to it is the buspirate used as ADC. The board on the bottom left is a
TTL-to-RS485 converter and the board in the middle is the unit under test.</figcaption>
</figure><p>The photodiode's photocurrent is converted into a voltage using a very simple transimpedance amplifier based around a
<a class="reference external" href="http://ww1.microchip.com/downloads/en/DeviceDoc/21733j.pdf">MCP6002</a> opamp that was damped into oblivion with a couple nanofarads of capacitance in its feedback loop. The <a class="reference external" href="http://ww1.microchip.com/downloads/en/DeviceDoc/21733j.pdf">MCP6002</a>
is a fine choice here since I had a bunch and because it is a CMOS opamp, meaning it has low bias current that would
mess up our measurements. For many applications, opamp bias current is not a big issue but when using the opamp to
directly measure very small currents at its input it quickly swamps out the signal for most BJT-input types.</p>
<p>The transimpedance amplifier's output is read from the computer using the ADC input of a buspirate USB thinggamajob. In
general I would not recommend the buspirate as a tool for this job since it's ADC is not particularly good and it's
programming interface is positively atrocious, but it was what I had and it beat first wiring up one of the dedicated
ADC chips I had in my parts bin.</p>
<p>The computer runs a small python script cycling the LED tape through all its BCM period settings and taking a brightness
measurement at each step. Later on, these measurements can be plotted to visualize the resulting slope's linearity, and
we can even do a simulation of the resulting brightness for all possible control values by just adding the measured
photocurrents for a certain BCM setpoint just as our retinas would do.</p>
<figure data-pagefind-ignore>
<img src="images/driver_linearity_raw.svg" alt="">
<figcaption>
A plot of the measured brightness of our LED tape for each BCM period. The brightness values are normalized
to the value measured at the LSB setpoint (brightness=1/65535). Ideally, this plot would show a straight
line with slope 1. Obviously, it doesn't. The bend in the curve is caused by the above-mentioned duty cycle
offset adding an offset to all brightness values. Shown is both the raw data (light), which has essentially zero
measurement error and a linear fit (dark).
The plot is in log-log to approximate how the human eye would perceive brightness, i.e. highly sensitive at
low values but not very sensitive at all at large values.
</figcaption>
</figure><p>While it would be possible to fully automate the optimization of BCM driver lookup tables, we needed only one and in the
end I just sat down and manually tweaked the ideal values we initially calculated until I liked the result. You can see
the resulting brightness curve below.</p>
<div class="subfigure" data-pagefind-ignore>
<figure>
<img src="images/uncorrected_brightness_sim.svg" alt="">
<figcaption>
Calculated brightness curve for the uncorrected BCM setup. As you can see, at low setpoints the result
is about as smooth as sandpaper, which is well in line with our observations. At high setpoints the
offset gets swamped out and the nonlinearity in the low bits is not visible anymore.
</figcaption>
</figure>
<figure>
<img src="images/corrected_brightness_sim.svg" alt="">
<figcaption>
Brightness curve for the corrected BCM setup extrapolated using actual measurements. Looks as buttery
smooth in real life as it does in this plot.
</figcaption>
</figcaption>
</figure>
</div></div>
</div>
<div class="section" id="controlling-the-driver">
<h3>Controlling the driver</h3>
<p>Now that our driver was behaving linear enough that you couldn't see it actually wasn't we needed a nice way to control
it from a computer of our choice. In the ultimate application (our staircase) we'll use a raspberry pi for this. Since
we already settled on an <a class="reference external" href="https://en.wikipedia.org/wiki/RS-485">RS485</a> bus for its robustness and simplicity, we had to device a protocol to control the driver
over this bus. Here, we settled on a simple, <a class="reference external" href="https://en.wikipedia.org/wiki/Consistent_Overhead_Byte_Stuffing">COBS</a>-based protocol for the reasons I wrote about in <a class="reference external" href="serial-protocols">How to talk to your
microcontroller over serial</a>.</p>
<p>To address our driver nodes, we modified the Makefile to build a random 32-bit MAC into each firmware image. The
protocol has only five message types:</p>
<ol class="arabic simple">
<li>A 0-byte <em>ping</em> packet, to which each node would reply with its own address in the
first 100ms after boot. This can be used to initially discover the addresses of all nodes connected to the bus. You'd
spam the bus with <em>ping</em> packets, and then hit reset on each node in turn. The control computer would then receive
each device's MAC address as you hit reset.</li>
<li>A 4-byte <em>address</em> packet that says which device that the following packet is for. This way of us using the packet
length instead of a packet type field is not particularly elegant, but our system is simple enough and it was easy to
implement.</li>
<li>A 64-byte <em>frame buffer</em> packet that contains 16 bits of left-aligned brightness data for every channel</li>
<li>A one-byte <em>get status</em> packet that tells the device to respond with...</li>
<li>...a 27-byte status packet containing a brief description of the firmware (version number, channel count, bit depth
etc.) as well as the device's current life stats (VCC, temperature, uptime, UART frame errors etc.).</li>
</ol>
<p>Wrapped up in a nice python interface we can now easily enumerate any drivers we connect to a bus, query their status
and control their outputs.</p>
</div>
<div class="section" id="conclusion">
<h3>Conclusion</h3>
<div class="subfigure" data-pagefind-ignore>
<figure>
<a href="images/olsndot_schematic.png">
<img src="images/olsndot_schematic.png" alt="A picture of the LED driver schematic">
</a>
<figcaption>The LED driver <a href="images/olsndot_schematic.png">schematic</a></figcaption>
</figure>
<figure>
<a href="images/olsndot_pcb.png">
<img src="images/olsndot_pcb.png" alt="A picture of the LED driver PCB layout">
</a>
<figcaption>The LED driver <a href="images/olsndot_pcb.png">PCB layout</a></figcaption>
</figure>
</div><p>Putting some thought into the control circuitry and software, you can easily control large numbers of channels of LEDs
using extremely inexpensive driving hardware without any compromises on dynamic range. The design we settled on can
drive 32 channels of LED tape with a dynamic range of 14bit at a BOM cost of below 10€. All it really takes is a couple
of shift registers and a mildly bored STM32 microcontroller.</p>
<p>Get a PDF file of the schematic and PCB layout <a class="reference external" href="olsndot_v02_schematics_and_pcb.pdf">here</a> or download the CAD files
and the firmware sources <a class="reference external" href="https://github.com/jaseg/led_drv">from github</a>. You can view the Jupyter notebook used to
analyze the brightness measurement data <a class="reference external" href="http://nbviewer.jupyter.org/github/jaseg/led_drv/blob/master/doc/Run_analysis.ipynb">here</a>.</p>
</div>
</div>
</div>
</main><footer>
Copyright © 2025 Jan Sebastian Götte
/ <a href="/about/">About</a>
/ <a href="/imprint/">Imprint</a>
</footer>
<script type="text/javascript" src="/pagefind/pagefind-ui.js" defer></script>
<script>
window.addEventListener('DOMContentLoaded', (event) => {
new PagefindUI({element: "#search", showSubResults: true});
});
</script>
<script type="speculationrules">
{
"prerender": [
{
"source": "document",
"where": {
"and": [
{"href_matches": "/*"}
]
},
"eagerness": "moderate"
}
]
}
</script>
</body>
</html>

0
content/blog/multichannel-led-driver/olsndot_v02_schematics_and_pcb.pdf → blog/multichannel-led-driver/olsndot_v02_schematics_and_pcb.pdf
View file

0
content/blog/paper-sampling-mesh-monitor/fig_edge_risetime.pdf → blog/paper-sampling-mesh-monitor/fig_edge_risetime.pdf
View file

0
content/blog/paper-sampling-mesh-monitor/fig_edge_risetime.png → blog/paper-sampling-mesh-monitor/fig_edge_risetime.png
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 87 KiB

After

Width:  |  Height:  |  Size: 87 KiB

Before After
Before After

147
blog/paper-sampling-mesh-monitor/index.html Normal file
View file

@ -0,0 +1,147 @@
<!DOCTYPE html>
<html><head>
<meta charset="utf-8">
<title>New paper: Monitoring Tamper-Sensing Meshes Using Low-Cost, Embedded Time-Domain Reflectometry | Home</title>
<meta name="description" content="">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="mobile-web-app-capable" content="yes">
<meta name="color-scheme" content="dark light">
<link rel="stylesheet" href="/style.css">
<link rel="preload" href="/fonts/roboto_slab/RobotoSlab-VariableFont_wght.ttf" as="font" type="font/ttf" crossorigin />
<link rel="preload" href="/fonts/nyght-serif-main/fonts/WEB/NyghtSerif-Regular.woff2" as="font" type="font/woff2" crossorigin />
<link rel="preload" href="/fonts/nyght-serif-main/fonts/WEB/NyghtSerif-Bold.woff2" as="font" type="font/woff2" crossorigin />
<link rel="preload" href="/fonts/nyght-serif-main/fonts/WEB/NyghtSerif-BoldItalic.woff2" as="font" type="font/woff2" crossorigin />
</head>
<body><nav>
<div class="internal">
<a href="/" title="Home">Home</a>
<a href="/blog/" title="Blog">Blog</a>
<a href="/projects/" title="Projects">Projects</a>
<a href="/about/" title="About">About</a>
</div>
<div class="search">
<div id="search"></div>
</div>
<div class="external">
<a href="https://git.jaseg.de/" title="cgit">cgit</a>
<a href="https://github.com/jaseg" title="Github">Github</a>
<a href="https://gitlab.com/neinseg" title="Gitlab">Gitlab</a>
<a href="https://chaos.social/@jaseg" title="Mastodon">Mastodon</a>
</span>
</nav>
<header>
<h1>New paper: Monitoring Tamper-Sensing Meshes Using Low-Cost, Embedded Time-Domain Reflectometry</h1>
<ul class="breadcrumbs">
<li><a href="/">jaseg.de</a></li>
<li><a href="/blog/">Blog</a></li><li><a href="/blog/paper-sampling-mesh-monitor/">New paper: Monitoring Tamper-Sensing Meshes Using Low-Cost, Embedded Time-Domain Reflectometry</a></li>
</ul>
<strong>2025-10-20</strong>
</header>
<main data-pagefind-body>
<div class="document">
<figure data-pagefind-ignore>
<img src="pic_board_setup_2_small.jpg" alt="A PCB with several chips sitting on a table with another PCB
with only traces on it plugged in through a board-edge connector. The first PCB looks not very complex.">
<figcaption>
The final setup. On the right is the measurement board, and on the left is the mesh test specimen plugged
in. In a real application, you would integrate both into your target circuit.
</figcaption>
</figure><p>I've got a new paper accepted at CHES, to be published in TCHES 2026/1 around beginning of December and out
<a class="reference external" href="https://eprint.iacr.org/2025/1962">on eprint now</a>. The topic of the paper is a way of monitoring a tamper-sensing
mesh through time-domain reflectometry using very cheap components. The end result is a circuit that costs about 10 € in
parts that is able to measure TDR responses with a few hundred picoseconds of resolution.</p>
<p>Tamper-Sensing meshes are squiggly circuit traces that are used to tamper-proof high-security devices like hardware
security modules, ATM pin pads and countertop card payment terminals. Any area where you would like to prevent an
attacker from drilling or sawing through in a physical attack, you completely cover with one or more such circuit traces
in a meandering pattern. I've written up some work on a KiCad plugin for creating these meshes <a class="reference external" href="http://jaseg.de/blog/kicad-mesh-plugin/">in another post</a>.</p>
<p>Up to now, the state of the art in monitoring these security meshes has mostly been finding ways to precisely monitor
their ohmic resistance in the analog domain. This has the disadvantage of both being fairly complex in circuitry and of
presenting a steep trade-off between sensitivity and false-positive rate since all you get out of the whole mesh is a
single analog measurement containing maybe 12 to 16 bits of entropy. There have been a few papers on using more advanced
RF techniques, but they all either required really expensive circuitry and/or highly customized meshes that for instance
couldn't easily be fitted into arbitrary shapes.</p>
<figure data-pagefind-ignore>
<img src="fig_edge_risetime.png" alt="Four plots showing edge response for four different chips: 74LVC2G157,
MAX3748, TDP0604 and PI3HDX12211. The first two are fairly slow at about 1 ns risetime, while the last two are
very fast at around 300 ps risetime.">
<figcaption>
The sampling edges as measured by the board itself. As you can see, using a cheap microcontroller and some
cheap display signal redriver ICs along with commodity RF schottkies you can get pretty spicy edges on a
budget. <a href="fig_edge_risetime.pdf">Link to full resolution.</a>
</figcaption>
</figure><p>In this paper, I wrote up a method using the high-resolution timer of an inexpensive <a class="reference external" href="https://www.st.com/resource/en/datasheet/stm32g474cb.pdf">STM32G4-series microcontroller</a> together with a DisplayPort/HDMI &quot;redriver&quot; chips meant for
amplifying high-speed display signals to create fast pulse edges. I characterized several chips, with the best
performers being TI's <a class="reference external" href="https://www.ti.com/product/TDP0604">TDP0604</a> and Diodes' <a class="reference external" href="https://www.diodes.com/part/view/PI3HDX12211">PI3HDX12211</a>, coming in at 2 to 5 € depending on where and how much you buy. The
fast edges generated by these drivers are then fed to a set of four-diode sampling gates using cheap RF schottky diodes
to create a really cheap but fast time-domain reflectometer. Using this TDRD circuit, a security mesh can be monitored
much more precisely than before, since the circuit creates a sort of fingerprint of the mesh's trace along its length.</p>
<div class="subfigure" data-pagefind-ignore>
<figure>
<img src="pic_74lvc_small.jpg" alt="">
<figcaption>
74LVC2G157
</figcaption>
</figure>
<figure>
<img src="pic_max3748_small.jpg" alt="">
<figcaption>
MAX3748
</figcaption>
</figure>
<figure>
<img src="pic_tdp0604_small.jpg" alt="">
<figcaption>
TDP0604
</figcaption>
</figure>
<figure>
<img src="pic_pi3hdx_small.jpg" alt="">
<figcaption>
PI3HDX12211
</figcaption>
</figure>
</div><p>One of the fun highlights of this project to me was micro-soldering test boards using different redriver ICs. Above, you
can see the result of that soldering work. I was really happy with my cheap aliexpress microscope and with my fancy
titanium tweezers!</p>
<p>Have a look into the paper, where I wrote up details on the circuitry as well as a whole bunch of (&gt;1000!) measurements
characterizing the system. As it turns out, it's really sensitive to attacks while being reasonably robust to
environmental disturbances. In fact, it's so sensitive that the circuit can distinguish multiple identical (!) copies of
the same mesh produces by JLCPCB from their manufacturing tolerances such as FR-4 fiber weave alignment.</p>
<p>You can find a preprint of the paper <a class="reference external" href="https://eprint.iacr.org/2025/1962">on eprint</a>, and I'll update this post with a
link to the published version of the paper when it becomes available. The eprint is identical to the published version
as of now.</p>
<p>The source code of the project is available at <a class="reference external" href="https://git.jaseg.de/sampling-mesh-monitor.git">https://git.jaseg.de/sampling-mesh-monitor.git</a>.</p>
</div>
</main><footer>
Copyright © 2025 Jan Sebastian Götte
/ <a href="/about/">About</a>
/ <a href="/imprint/">Imprint</a>
</footer>
<script type="text/javascript" src="/pagefind/pagefind-ui.js" defer></script>
<script>
window.addEventListener('DOMContentLoaded', (event) => {
new PagefindUI({element: "#search", showSubResults: true});
});
</script>
<script type="speculationrules">
{
"prerender": [
{
"source": "document",
"where": {
"and": [
{"href_matches": "/*"}
]
},
"eagerness": "moderate"
}
]
}
</script>
</body>
</html>

0
content/blog/paper-sampling-mesh-monitor/pic_74lvc_small.jpg → blog/paper-sampling-mesh-monitor/pic_74lvc_small.jpg
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 100 KiB

After

Width:  |  Height:  |  Size: 100 KiB

Before After
Before After

0
content/blog/paper-sampling-mesh-monitor/pic_board_setup_2_small.jpg → blog/paper-sampling-mesh-monitor/pic_board_setup_2_small.jpg
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 744 KiB

After

Width:  |  Height:  |  Size: 744 KiB

Before After
Before After

0
content/blog/paper-sampling-mesh-monitor/pic_max3748_small.jpg → blog/paper-sampling-mesh-monitor/pic_max3748_small.jpg
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 104 KiB

After

Width:  |  Height:  |  Size: 104 KiB

Before After
Before After

0
content/blog/paper-sampling-mesh-monitor/pic_pi3hdx_small.jpg → blog/paper-sampling-mesh-monitor/pic_pi3hdx_small.jpg
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 104 KiB

After

Width:  |  Height:  |  Size: 104 KiB

Before After
Before After

0
content/blog/paper-sampling-mesh-monitor/pic_tdp0604_small.jpg → blog/paper-sampling-mesh-monitor/pic_tdp0604_small.jpg
View file

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 100 KiB

After

Width:  |  Height:  |  Size: 100 KiB

Before After
Before After

101
blog/private-contact-discovery/index.html Normal file
View file

@ -0,0 +1,101 @@
<!DOCTYPE html>
<html><head>
<meta charset="utf-8">
<title>Private Contact Discovery | Home</title>
<meta name="description" content="">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="mobile-web-app-capable" content="yes">
<meta name="color-scheme" content="dark light">
<link rel="stylesheet" href="/style.css">
<link rel="preload" href="/fonts/roboto_slab/RobotoSlab-VariableFont_wght.ttf" as="font" type="font/ttf" crossorigin />
<link rel="preload" href="/fonts/nyght-serif-main/fonts/WEB/NyghtSerif-Regular.woff2" as="font" type="font/woff2" crossorigin />
<link rel="preload" href="/fonts/nyght-serif-main/fonts/WEB/NyghtSerif-Bold.woff2" as="font" type="font/woff2" crossorigin />
<link rel="preload" href="/fonts/nyght-serif-main/fonts/WEB/NyghtSerif-BoldItalic.woff2" as="font" type="font/woff2" crossorigin />
</head>
<body><nav>
<div class="internal">
<a href="/" title="Home">Home</a>
<a href="/blog/" title="Blog">Blog</a>
<a href="/projects/" title="Projects">Projects</a>
<a href="/about/" title="About">About</a>
</div>
<div class="search">
<div id="search"></div>
</div>
<div class="external">
<a href="https://git.jaseg.de/" title="cgit">cgit</a>
<a href="https://github.com/jaseg" title="Github">Github</a>
<a href="https://gitlab.com/neinseg" title="Gitlab">Gitlab</a>
<a href="https://chaos.social/@jaseg" title="Mastodon">Mastodon</a>
</span>
</nav>
<header>
<h1>Private Contact Discovery</h1>
<ul class="breadcrumbs">
<li><a href="/">jaseg.de</a></li>
<li><a href="/blog/">Blog</a></li><li><a href="/blog/private-contact-discovery/">Private Contact Discovery</a></li>
</ul>
<strong>2019-06-22</strong>
</header>
<main data-pagefind-body>
<div class="document">
<p>Private Contact Discovery (PCD) is the formal name for the problem modern smartphone messenger applications have on
installation: Given a user's address book, find out which of their contacts also use the same messenger without the
messenger's servers learning anything about the user's address book. The widespread non-private way to do this is to
simply upload the user's address book to the app's operator's servers and do an SQL JOIN keyed on the phone number field
against the database of registered users. People have tried sprinkling some hashes over these phone numbers in an
attempt to improve privacy, but obviously running a brute-force preimage attack given a domain of maybe a few billion
valid inputs is not cryptographically hard.</p>
<p>Private Contact Discovery can be phrased in terms of Private Set Intersection (PSI), the cryptographic problem of having
two parties holding one set each find the intersection of their sets without disclosing any other information. PSI has
been an active field of research for a while and already yielded useful results for some use cases. Alas, none of those
results were truly practical yet for usage in PCD in a typical messenger application. They would require too much CPU
time or too much data to be transferred.</p>
<p>At USENIX Security 2019, Researchers from technical universities Graz and Darmstadt published a paper titled <em>Private
Contact Discovery at Scale</em>
(<a class="reference external" href="https://eprint.iacr.org/2019/517">eprint</a> | <a class="reference external" href="https://eprint.iacr.org/2019/517.pdf">PDF</a>).
In this paper, they basically optimize the hell out of existing cryptographic solutions to private contact discovery,
jumping from a still-impractical state of the art right to practicality. Their scheme allows a client with 1k contacts
to run PCD against a server with 1B contacts in about 3s on a phone. The main disadvantage of their scheme is that it
requires the client to in advance download a compressed database of all users, that clocks in at about 1GB for 1B users.</p>
<p>I found this paper very interesting for its immediate practical applicability. As an excuse to dig into the topic some
more, I gave a short presentation at my university lab's research seminar on this paper
(slides: <a class="reference external" href="mori_semi_psi_talk.pdf">PDF</a> | <a class="reference external" href="mori_semi_psi_talk.odp">ODP</a>).</p>
<p>Even if you're not working on secure communication systems on a day-to-day basis this paper might interest you. If
you're working with social account information of any kind I can highly recommend giving it a look. Not only might your
users benefit from improved privacy, but your company might be able to avoid a bunch of data protection and
accountability issues by simply not producing as much sensitive data in the first place.</p>
</div>
</main><footer>
Copyright © 2025 Jan Sebastian Götte
/ <a href="/about/">About</a>
/ <a href="/imprint/">Imprint</a>
</footer>
<script type="text/javascript" src="/pagefind/pagefind-ui.js" defer></script>
<script>
window.addEventListener('DOMContentLoaded', (event) => {
new PagefindUI({element: "#search", showSubResults: true});
});
</script>
<script type="speculationrules">
{
"prerender": [
{
"source": "document",
"where": {
"and": [
{"href_matches": "/*"}
]
},
"eagerness": "moderate"
}
]
}
</script>
</body>
</html>

0
content/blog/private-contact-discovery/mori_semi_psi_talk.odp → blog/private-contact-discovery/mori_semi_psi_talk.odp
View file

0
content/blog/private-contact-discovery/mori_semi_psi_talk.pdf → blog/private-contact-discovery/mori_semi_psi_talk.pdf
View file

272
blog/serial-protocols/index.html Normal file
View file

@ -0,0 +1,272 @@
<!DOCTYPE html>
<html><head>
<meta charset="utf-8">
<title>How to talk to your microcontroller over serial | Home</title>
<meta name="description" content="">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="mobile-web-app-capable" content="yes">
<meta name="color-scheme" content="dark light">
<link rel="stylesheet" href="/style.css">
<link rel="preload" href="/fonts/roboto_slab/RobotoSlab-VariableFont_wght.ttf" as="font" type="font/ttf" crossorigin />
<link rel="preload" href="/fonts/nyght-serif-main/fonts/WEB/NyghtSerif-Regular.woff2" as="font" type="font/woff2" crossorigin />
<link rel="preload" href="/fonts/nyght-serif-main/fonts/WEB/NyghtSerif-Bold.woff2" as="font" type="font/woff2" crossorigin />
<link rel="preload" href="/fonts/nyght-serif-main/fonts/WEB/NyghtSerif-BoldItalic.woff2" as="font" type="font/woff2" crossorigin />
</head>
<body><nav>
<div class="internal">
<a href="/" title="Home">Home</a>
<a href="/blog/" title="Blog">Blog</a>
<a href="/projects/" title="Projects">Projects</a>
<a href="/about/" title="About">About</a>
</div>
<div class="search">
<div id="search"></div>
</div>
<div class="external">
<a href="https://git.jaseg.de/" title="cgit">cgit</a>
<a href="https://github.com/jaseg" title="Github">Github</a>
<a href="https://gitlab.com/neinseg" title="Gitlab">Gitlab</a>
<a href="https://chaos.social/@jaseg" title="Mastodon">Mastodon</a>
</span>
</nav>
<header>
<h1>How to talk to your microcontroller over serial</h1>
<ul class="breadcrumbs">
<li><a href="/">jaseg.de</a></li>
<li><a href="/blog/">Blog</a></li><li><a href="/blog/serial-protocols/">How to talk to your microcontroller over serial</a></li>
</ul>
<strong>2018-05-19</strong>
</header>
<main data-pagefind-body>
<div class="document">
<p>Scroll to the end for the <a class="reference internal" href="#conclusion">TL;DR</a>.</p>
<p>In this article I will give an overview on the protocols spoken on serial ports, highlighting common pitfalls. I will
summarize some points on how to design a serial protocol that is simple to implement and works reliably even under error
conditions.</p>
<p>If you have done low-level microcontroller firmware you will regularly have had to stuff some data up a serial port to
another microcontroller or to a computer. In the age of USB, an old-school serial port is still the simplest and
quickest way to get communication to a control computer up and running. Integrating a ten thousand-line USB stack into
your firmware and writing the necessary low-level drivers on the host side might take days. Poking a few registers to
set up your UART to talk to an external hardware USB to serial converter is a matter of minutes.</p>
<p>This simplicity is treacherous, though. Oftentimes, you start writing your serial protocol as needs arise. Things might
start harmless with something like <tt class="docutils literal">SET_LED ON\n</tt>, but as the code grows it is easy to end up in a hot mess of command
modes, protocol states that breaks under stress. The ways in which serial protocols break are manifold. The simplest one
is that at some point a character is mangled, leading to both ends of the conversation ending up in misaligned protocol
states. With a fragile protocol, you might end up in a state that is hard to recover from. In extreme cases, this leads
to code such as <a class="reference external" href="https://github.com/juhasch/pyBusPirateLite/blob/dece35f6e421d4f6a007d1db98d148e2f2126ebb/pyBusPirateLite/base.py#L113">this gem</a> performing some sort of arcane ritual to get back to some known state, and all just because
someone did not do their homework. Below we'll embark on a journey through the lands of protocol design, exploring the
facets of this deceptively simple problem.</p>
<div class="section" id="text-based-serial-protocols">
<h2>Text-based serial protocols</h2>
<p>The first serial protocol you've likely written is a human-readable, text-based one. Text-based protocols have the big
advantage that you can just print them on a terminal and you can immediately see what's happening. In most cases you can
even type out the protocol with your bare hands, meaning that you don't really need a debugging tool beyond a serial
console.</p>
<p>However, text-based protocols also have a number of disadvantages. Depending on your application, these might not matter
and in many cases a text-based protocol is the most sensible solution. But then, in some cases they might and it's good
to know when you hit one of them.</p>
<div class="section" id="problems">
<h3>Problems</h3>
<div class="section" id="low-information-density">
<h4>Low information density</h4>
<p>Generally, you won't be able to stuff much more than four or five bit of information down a serial port using a
single byte of a human-readable protocol. In many cases you will get much less. If you have 10 commands that are only
issued a couple times a second nobody cares that you spend maybe ten bytes per command on nice, verbose strings such as
<tt class="docutils literal">SET LED</tt>. But if you're trying to squeeze a half-kilobyte framebuffer down the line you might start to notice the
difference between hex and base-64 encoding, and a binary protocol might really be more up to the job.</p>
</div>
<div class="section" id="complex-parsing-code">
<h4>Complex parsing code</h4>
<p>On the computer side of thing, with the whole phalanx of an operating system, the standard library of your programming
language of choice and for all intents and purposes unlimted CPU and memory resources to spare you can easily parse
anything spoken on a serial port in real time, even at a blazing fast full Megabaud. The microcontroller side however is
an entirely different beast. On a small microcontroller, <a class="reference external" href="http://git.musl-libc.org/cgit/musl/tree/src/stdio/vfprintf.c">printf</a> alone will eat about half your flash. On most small
microcontrollers, you just won't get a regex library even though it would make parsing textual commands <em>so much
simpler</em>. Lacking these resources, you might end up hand-knitting a lot of low-level C code to do something seemingly
simple such as parsing <tt class="docutils literal">set_channel (13, <span class="pre">1.1333)\n</span></tt>. These issues have to be taken into account in the protocol design
from the beginning. For example, you don't really need matching parentheses, don't use them.</p>
</div>
<div class="section" id="fragile-protocol-state">
<h4>Fragile protocol state</h4>
<p>Say you have a <tt class="docutils literal">SET_DISPLAY</tt> command. Now say your display can display four lines of text. The obvious approach to this
is probably the <a class="reference external" href="https://en.wikipedia.org/wiki/Simple_Mail_Transfer_Protocol">SMTP</a> or <a class="reference external" href="https://en.wikipedia.org/wiki/Hypertext_Transfer_Protocol">HTTP</a> way of sending <tt class="docutils literal">SET_DISPLAY\nThis is line 1\nThis is line 2\n\n</tt>. This would certainly
work, but it is very fragile. With this protocol, you're in trouble if at any point the terminating second newline
character gets mangled (say, someone unplugs the cable, or the control computer reboots, or a cosmic ray hits something
and <tt class="docutils literal">0x10 '\n'</tt> turns into <tt class="docutils literal">0x50 'P'</tt>).</p>
</div>
<div class="section" id="timeouts-don-t-work">
<h4>Timeouts don't work</h4>
<p>You might try to solve the problem of your protocol state machine tangling up with a timeout. &quot;If I don't get a valid
command for more than 200ms I go back to default state.&quot; But consider the above example. Say, your control computer
sends a <tt class="docutils literal">SET_DISPLAY</tt> command every 100ms. If in one of them the state machine tangles up, the parser hangs since the
timeout is never hit, because a new line of text is arriving every 100ms.</p>
</div>
<div class="section" id="framing-is-hard">
<h4>Framing is hard</h4>
<p>You might also try to drop the second newline and using a convention such as <tt class="docutils literal">SET_DISPLAY</tt> is followed by two lines of
text, then commands resume.&quot;. This works as long as your display contents never look like commands. If you are only ever
displaying the same three messages on a character LCD that might work, but if you're displaying binary framebuffer
data you've lost.</p>
</div>
</div>
<div class="section" id="solutions">
<h3>Solutions</h3>
<div class="section" id="keep-the-state-machine-simple">
<h4>Keep the state machine simple</h4>
<p>In a text-based protocol, always use a single line of text to represent a single command. Don't do protocol states or
modes where you can toggle between different interpretations for a line. If you have to send human-readable text as part
of a command (such as <tt class="docutils literal">SET_DISPLAY</tt>), escape it so it doesn't contain any newlines.</p>
<p>This way, you keep your protocol state machine simple. If at any time your serial trips and flips a bit or looses a byte
your protocol will recover on the next newline character, returning to its base state.</p>
</div>
<div class="section" id="encode-numbers-in-hex-when-possible">
<h4>Encode numbers in hex when possible</h4>
<p>Printing a number in hexadecimal is a very tidy operation, even on the smalest 8-bit microcontrollers. In contrast,
printing decimal requires both division and remainder in a loop which might get annoyingly code- and time-intensive on
large numbers (say a 32-bit int) and small microcontrollers.</p>
<p>If you have to send fractional values, consider their precision. Instead of sending a 12 bit ADC result as a 32-bit
float formatted like <tt class="docutils literal">0.176513671875</tt> sending <tt class="docutils literal">0x2d3</tt> and dividing by 4096 on the host might be more sensible. If you
really have to communicate big floats and you can't take the overhead of including both <a class="reference external" href="http://git.musl-libc.org/cgit/musl/tree/src/stdio/vfprintf.c">printf</a> and <a class="reference external" href="http://git.musl-libc.org/cgit/musl/tree/src/stdio/vfscanf.c">scanf</a> you can
use hexadecimal floating point, which is basically <tt class="docutils literal"><span class="pre">hex((int)foo)</span> + &quot;.&quot; + <span class="pre">hex((int)(65536*(foo</span> - <span class="pre">(int)foo)))</span></tt> for four
digits. You can also just hex-encode the binary <a class="reference external" href="https://en.wikipedia.org/wiki/IEEE_754">IEEE-754</a> representation of the float, sending <tt class="docutils literal"><span class="pre">hex(*(int</span> <span class="pre">*)&amp;float)</span></tt>.
Most programming languages will have a <a class="reference external" href="https://docs.python.org/3.5/library/struct.html">simple, built-in means to parse this sort of thing</a>.</p>
</div>
<div class="section" id="escape-multiline-strings">
<h4>Escape multiline strings</h4>
<p>If you have to send arbitrary strings, escape special characters. This not only has the advantage of yielding a robust
protocol: It also ensures you can actually see everything that's going on when debugging. The string <tt class="docutils literal">&quot;\r\n&quot;</tt> is easy to
distinguish from <tt class="docutils literal">&quot;\n&quot;</tt> while your terminal emulator might not care.</p>
<p>The simplest encoding to use is the C-style backslash encoding. Host-side, most languages will have a <a class="reference external" href="https://docs.python.org/3.5/library/codecs.html#text-encodings">built-in means of
escaping a string like that</a>.</p>
</div>
</div>
<div class="section" id="encoding-binary-data">
<h3>Encoding binary data</h3>
<p>For binary data, hex and base-64 are the most common encodings. Since hex is simpler to implement I'd go with it unless
I really need the 30% bandwidth improvement base-64 brings.</p>
</div>
</div>
<div class="section" id="binary-serial-protocols">
<h2>Binary serial protocols</h2>
<p>In contrast to anything human-readable, binary protocols are generally more bandwidth-efficient and are easier to format
and parse. However, binary protocols come with their own version of the caveats we discussed for text-based protocols.</p>
<div class="section" id="the-framing-problem-in-binary-protocols">
<h3>The framing problem in binary protocols</h3>
<p>The most basic problems with binary protocols as with text-based ones is framing, i.e. splitting up the continuous
serial data stream into discrete packets. The issue is that it is that you have to somehow mark boundaries between
frames. The simplest way would be to use some special character to delimit frames, but then any 8-bit character you
could choose could also occur within a frame.</p>
<div class="section" id="slip-ppp-like-special-character-framing">
<h4>SLIP/PPP-like special character framing</h4>
<p>Some protocols solve this problem much like we have solved it above for strings in line-based protocols, by escaping any
occurence of the special delimiter character within frames. That is, if you want to use <tt class="docutils literal">0x00</tt> as a delimiter, you would
encode a packet containing <tt class="docutils literal">0xde 0xad 0x00 0xbe 0xef</tt> as something like <tt class="docutils literal">0xde 0xad 0x01 0x02 0xbe 0xef</tt>, replacing the
null byte with a magic sequence. This framing works, but is has one critical disadvantage: The length of the resulting
escaped data is dependent on the raw data, and in the worst case twice as long. In a raw packet consisting entirely of
null bytes, every byte must be escaped with two escape bytes. This means that in this case the packet length doubles,
and in this particular case we're even less efficient than base-64.</p>
<p>Highly variable packet length is also bad since it makes it very hard to make any timing guarantees for our protocol.</p>
</div>
<div class="section" id="bit-framing">
<h4>9-bit framing</h4>
<p>A framing mode sometimes used is to configure the UARTs to transmit 9-bit characters and to use the 9th bit to designate
control characters. This works really well, and gives plenty of control characters to work with. The main problem with
this is that a 9-bit serial interface is highly nonstandard and you need UARTs on both ends that actually support this
mode. Another issue is that though more efficient than both delmitier-based and purely text-based protocols, it still
incurs an extra about 10% of bandwidth overhead. This is not a lot if all you're sending is a little command every now
and then, but if you're trying to push large amounts of data through your serial it's still bad.</p>
</div>
<div class="section" id="cobs">
<h4>COBS</h4>
<p>Given the limitations of the two above-mentioned framing formats, we really want something better. The <a class="reference external" href="https://en.wikipedia.org/wiki/Serial_Line_Internet_Protocol">Serial Line
Internet Protocol (SLIP)</a> as well as the <a class="reference external" href="https://en.wikipedia.org/wiki/Point-to-Point_Protocol">Point to Point Protocol (PPP)</a>, standardized in 1988 and 1994 respectively,
both use escape sequences. This might come as a surprise, but humanity has actually still made significant technological
progress on protocols for 8-bit serial interfaces until the turn of the millennium. In 1999, <a class="reference external" href="http://www.stuartcheshire.org/papers/COBSforToN.pdf">Consistent Overhead Byte
Stuffing (COBS)</a> (<a class="reference external" href="https://en.wikipedia.org/wiki/Consistent_Overhead_Byte_Stuffing">wiki</a>) was published by a few
researchers from Apple Computer and Stanford University. As a reaction on the bandwidth doubling problem present in
<a class="reference external" href="https://en.wikipedia.org/wiki/Point-to-Point_Protocol">PPP</a>, COBS <em>always</em> has an overhead of a single byte, no matter what or how long a packet's content is.</p>
<p>COBS uses the null byte as a delimiter interleaves all the raw packet data and a <a class="reference external" href="https://en.wikipedia.org/wiki/Run-length_encoding">run-length encoding</a> of the non-zero
portions of the raw packet. That is, it prepends the number of bytes until the first zero byte to the packet, plus one.
Then it takes all the leading non-zero bytes of the packet, unmodified. Then, it again encodes the distance from the
first zero to the second zero, plus one. And then it takes the second non-zero run of bytes unmodified. And so on. At
the end, the packet is terminated with a zero byte.</p>
<p>The result of this scheme is that the encoded packet does not contain any zero bytes, as every zero byte has been
replaced with the number of bytes until the next zero byte, plus one, and that can't be zero. Both formatter and parser
each have to keep a counter running to keep track of the distances between zero bytes. The first byte of the packet
initializes that counter and is dropped by the parser. After that, every encoded byte received results in one raw byte
parsed.</p>
<p>While this might sound more complicated than the escaping explained above, the gains in predictability and efficiency
are worth it. An implementation of encoder and decoder should each be about ten lines of C or two lines of Python. A
minor asymmetry of the protocol is that while decoding can be done in-place, encoding either needs two passes or you
need to scan forward for the next null byte.</p>
</div>
</div>
<div class="section" id="state-machines-and-error-recovery">
<h3>State machines and error recovery</h3>
<p>In binary protocols even more than in textual ones it is tempting to build complex state machines triggering actions on
a sequence of protocol packets. Please resist that temptation. As with textual protocols keeping the protocol state to
the minimum possible allows for a self-synchronizing protocol. A serial protocol should be designed such that if due to
a dropped packet or two both ends will naturally re-synchronize within another packet or two. A simple way of doing that
is to always transmit one semantic command per packet and to design these commands in the most <a class="reference external" href="https://en.wikipedia.org/wiki/Idempotence#Computer_science_meaning">idempotent</a> way possible.
For example, when filling a framebuffer piece by piece, include the offset in each piece instead of keeping track of it
on the receiving side.</p>
</div>
</div>
<div class="section" id="conclusion">
<h2>Conclusion</h2>
<p>Here's your five-step guide to serial bliss:</p>
<ol class="arabic simple">
<li>Unless you have super-special requirements, always use the slowest you can get away with from 9600Bd, 115200Bd or
1MBd. 8N1 framing if you're talking to anything but another microcontroller on the same board. Using common values
like these makes it easier when you'll inevitably have to guess these at some point in the future ;)</li>
<li>If you're doing something simple and speed is not a particular concern, use a human-readable text-based protocol. Use
one command/reply per line, begin each line with some sort of command word and format numbers in hexadecimal. Bonus
points for the device replying to unknown commands with a human-readable status message and printing a brief protocol
overview on boot.</li>
<li>If you're doing something even slightly nontrivial or need moderate throughput (&gt;1k commands per second or &gt;20 byte of
data per command) use a COBS-based protocol. A good starting point is a <tt class="docutils literal">[target <span class="pre">MAC][command</span> <span class="pre">ID][command</span>
arguments]</tt> packet format for multidrop busses. For single-drop you may decide to drop the MAC address.</li>
<li>Always include some sort of &quot;status&quot; command that prints life stats such as VCC, temperature, serial framing errors
and uptime. You'll need some sort of ping command anyway and that command might as well do something useful.</li>
<li>If at all possible, keep your protocol context-free across packets/lines. That is, a certain command should always be
self-contained, and no command should change the meaning of the next packet/line/command that is sent. This is really
important to allow for self-synchronization. If you really need to break up something into multiple commands, say you
want to set a large framebuffer in pieces, do it in a <a class="reference external" href="https://en.wikipedia.org/wiki/Idempotence#Computer_science_meaning">idempotent</a> way: Instead of sending something like <tt class="docutils literal">FRAMEBUFFER
<span class="pre">INCOMING:\n[byte</span> <span class="pre">0-16]\n[byte</span> <span class="pre">17-32]\n[...]\nEND</span> OF FRAME</tt> rather send <tt class="docutils literal">FRAMEBUFFER DATA FOR OFFSET 0: [byte
<span class="pre">0-16]\nFRAMEBUFFER</span> DATA FOR OFFSET 17: [byte <span class="pre">17-32]\n[...]\nSWAP</span> BUFFERS\n</tt>.</li>
</ol>
</div>
</div>
</main><footer>
Copyright © 2025 Jan Sebastian Götte
/ <a href="/about/">About</a>
/ <a href="/imprint/">Imprint</a>
</footer>
<script type="text/javascript" src="/pagefind/pagefind-ui.js" defer></script>
<script>
window.addEventListener('DOMContentLoaded', (event) => {
new PagefindUI({element: "#search", showSubResults: true});
});
</script>
<script type="speculationrules">
{
"prerender": [
{
"source": "document",
"where": {
"and": [
{"href_matches": "/*"}
]
},
"eagerness": "moderate"
}
]
}
</script>
</body>
</html>

Some files were not shown because too many files have changed in this diff Show more