Embarrassing Bugs That Made Better Guards

Some fixes start with an awkward mistake and end as a guard that keeps the next mistake quieter.

2026-05-29 · bluetape4k projects bug-fix note

While working on bluetape4k-projects, the bugs that hurt most are not always the clever ones. Sometimes they are the small assumptions that make you think: “why did I believe that?”

This post collects those cases. Not grand architecture, not heroic optimization. Just small mistakes that became issues, broke Nightly, and eventually turned into tests and release gates.

A candid write-up of five bluetape4k-projects fixes: null becoming zero, stale tests, cleanup timeouts, swallowed cancellation, and lost interrupt status.

How to Read This

Each case follows the same shape.

Question	Meaning
Why did I make the mistake?	The bad assumption at the time
How did we find it?	Symptom, failing log, round trip, Nightly signal
How did we fix it?	The actual change
How do we judge it?	Verification and remaining interpretation
What guard comes next?	How we avoid repeating the same mistake

1. An Empty Properties Value Became 0 Instead of null

This was a small, embarrassing bug. In Java Properties, a value like maxWaitMillis= reads to a human as “the value is empty.” But JavaPropsMapper read Properties values as untyped strings, then converted an empty string into 0 while binding to a nullable numeric Kotlin constructor parameter.

Why I Made the Mistake

I treated empty strings and numeric coercion as a generic Jackson problem. I assumed the “string to number” path would handle it well enough. But Java Properties has format-specific semantics. A property with no value and a property with value 0 mean very different things in configuration.

How We Found It

A round trip broke. The original model had a nullable numeric value as null, but after writing to Properties and reading back, it became 0. In datasource settings, values such as timeout, idle timeout, and max wait often use null to mean “use the default”; 0 can mean “set this to zero.”

The core verification was not fancy. We added a round trip containing empty Properties values and checked whether the parsed model matched the original model.

private val properties = """
|bluetape4k.datasources.default.maxWaitMillis=
|bluetape4k.datasources.read.idleTimeout=
""".trimMargin()

val parsedRoot = propsMapper.readValue<RootProperty>(properties)

parsedRoot.bluetape4k.datasources["default"] shouldBeEqualTo default
parsedRoot.bluetape4k.datasources["read"] shouldBeEqualTo read

How We Fixed It

The fix stayed inside JacksonText.Props.defaultMapper. Both Jackson 2 and Jackson 3 received numeric logical type coercion metadata, and the Properties-specific numeric module maps empty strings to null for boxed numeric types. Non-empty values still parse normally.

How We Judged It

Verification stayed format-specific. Four named datasource Properties round-trip tests passed for Jackson 2, four passed for Jackson 3, and the full Jackson 2/3 tests passed as well.

The important decision was not “fix CSV, TOML, and YAML the same way.” That would be another mistake. A bug found in Properties should not be blindly pushed into every text format, because each format has its own semantics.

Next Guard

The guard is simple: fix format-specific bugs in the format-specific mapper, and keep configuration round-trip tests around. In configuration files, null, an empty string, and 0 are different values.

2. What Nightly Taught Us: A Stale Test Is Also a Bug

Nightly run 26243476594 failed in three slices: IO HTTP, infra search-messaging, and Testcontainers graphdb-memgraph. At first glance, that kind of failure looks like external service instability. In reality, some tests had simply fallen behind the current code.

Why I Made the Mistake

I treated existing tests as existing verification. But tests are code too. If the DSL changes, tests can go stale. If Testcontainers reuse is enabled, CI credential state can go stale too.

One part of the fix updated MyBatis dynamic-sql join validation tests for the current on-based Kotlin DSL and current exception type. Another part disabled secured Elasticsearch singleton reuse to avoid stale CI credentials.

How We Found It

Nightly told us. A local targeted test can miss external-service slices that Full Nightly still exercises. The key point was not to patch production code immediately. The first question was: “does this test still represent the current API and environment?”

How We Fixed It

Stale test expectations were aligned with the current DSL and exception contract, and shared secured Elasticsearch test singleton reuse was disabled. Fast tests are good, but reusing credential state from a previous run is a bad release signal if it creates false failures.

How We Judged It

This was not a performance win or an elegant refactor. But in a release train, this kind of fix can matter more. The lesson was to treat a Nightly failure as evidence, not as “probably flaky.”

Next Guard

When an external-service test fails, check three things first.

• Does the test still follow the current public API and exception contract?
• Is Testcontainers reuse polluting CI state?
• Is the failure service readiness, client cleanup, credential reuse, or something else?

3. Memgraph Did Not Hang in the Query. It Hung in close()

Full Nightly timed out in MemgraphServerTest. The natural first suspects were container startup, readiness, or the image version. But the Bolt query path had already succeeded. The hang was inside Neo4j Java Driver’s Driver.close().

Why I Made the Mistake

It is easy to think, “a Testcontainers timeout must be a container problem.” Here, the container was not the problem. Client cleanup was holding the runner after a successful query.

How We Found It

The stack told the story. The failing point was not container startup and not the Bolt query. It was inside the driver close path. Without that clue, we could have wasted time changing image tags or extending readiness waits.

How We Fixed It

The Memgraph image and Neo4j driver version stayed as they were. They were already on the current line at the time. Instead, the test driver was restricted to a single connection pool/event-loop, telemetry and auto-commit retry were disabled, and blocking Driver.close() was replaced by closeAsync() with a bounded timeout.

The cleanup guard had to be explicit. If the query passed, the release-gate assertion had done its job. Driver cleanup must not hold the Nightly runner forever.

private fun Driver.closeWithin(timeout: Long, unit: TimeUnit) {
    val closeFuture = closeAsync().toCompletableFuture()
    try {
        closeFuture.get(timeout, unit)
    } catch (e: TimeoutException) {
        closeFuture.cancel(true)
        log.warn(e) { "Neo4j Driver close timed out after Memgraph query verification." }
    } catch (e: InterruptedException) {
        Thread.currentThread().interrupt()
        throw e
    }
}

The release-gate assertion remained: does the Bolt query succeed? Cleanup should be bounded.

How We Judged It

The single MemgraphServerTest had 6 passing tests, and the graphdb slice had 27 passing tests. The fix was not “wait longer”; it was “do not let cleanup hold the release gate hostage.”

Next Guard

When a Testcontainers-backed driver test succeeds in query verification but hangs in cleanup, do not start by changing readiness or image versions. Look at the close stack first, and keep a bounded cleanup path.

4. catch (Throwable) Turned Coroutine Cancellation into HTTP Failure

The Vert.x route coroutine helper forwarded failures from a suspend handler to RoutingContext.fail(...). That is fine for ordinary exceptions. The problem was that a broad Throwable catch also caught CancellationException.

Why I Made the Mistake

In an HTTP route, “send handler failures to fail” sounds reasonable. But coroutine cancellation is not an ordinary failure. In structured concurrency, cancellation is a control signal. If it is converted into an HTTP failure, a caller-cancelled operation starts looking like an application error.

How We Found It

The catch boundary was too broad. The helper put ordinary exceptions and cancellation into the same bucket.

How We Fixed It

Both suspendHandler and suspendFailureHandler now catch CancellationException first and rethrow it. Only ordinary exceptions are sent to ctx.fail(e). Regression tests lock both paths: cancellation and normal exception.

The rule is one line: do not send cancellation to the failure handler.

try {
    requestHandler(ctx)
} catch (e: CancellationException) {
    throw e
} catch (e: Throwable) {
    ctx.fail(e)
}

The tests keep the paths separate.

verify(exactly = 0) { ctx.fail(any<Throwable>()) } // cancellation
verify(exactly = 1) { ctx.fail(failure) }          // normal exception

How We Judged It

Behavior for ordinary handler exceptions stayed the same. They still flow through Vert.x failure handling. The only change is that cancellation is no longer disguised as failure.

Next Guard

Whenever a suspend boundary uses a broad catch, ask:

What happens if CancellationException is caught here?

If the answer is vague, the code is already risky. Rethrow CancellationException first or use a helper such as runCatchingNonCancellation.

5. runCatching Made Interrupts Look Like Timeouts

The gRPC server shutdown code used runCatching { awaitTermination(...) }.getOrDefault(false). It looked neat. If graceful termination failed or timed out, return false and proceed to shutdownNow().

But awaitTermination(...) can throw InterruptedException. runCatching swallowed that interrupt as just another failure that could become false. The caller thread’s interrupt status disappeared.

Why I Made the Mistake

runCatching is convenient. It makes lifecycle code look like “if it fails, return false.” But interruption from a blocking API is not an ordinary failure. It is a signal that must be preserved on the thread.

How We Found It

While reviewing shutdown lifecycle issues, we separated timeout from interruption. Both can look like graceful termination failure, but interruption carries state that must be returned to the caller.

How We Fixed It

Both the port-based server and the in-process server were changed to explicit try/catch. When InterruptedException is caught, the code restores the interrupt flag with Thread.currentThread().interrupt() and returns false. That still triggers shutdownNow(), but the caller’s interrupt status survives.

After forced shutdown, bounded termination waits were added so cleanup is not fire-and-forget.

Here, preserving the signal mattered more than making the code look tidy.

private fun awaitTerminationOrRestoreInterrupt(): Boolean =
    try {
        server.awaitTermination(SHUTDOWN_TIMEOUT_SECONDS, TimeUnit.SECONDS)
    } catch (e: InterruptedException) {
        Thread.currentThread().interrupt()
        false
    }

Regression tests make sure timeout and interruption are not treated as the same thing.

server.stop()

recordingServer.shutdownCalls shouldBeEqualTo 1
recordingServer.awaitTimedCalls shouldBeEqualTo 1
recordingServer.shutdownNowCalls shouldBeEqualTo 1
Thread.currentThread().isInterrupted.shouldBeTrue()

How We Judged It

Existing timeout behavior stayed the same. The changed part is interruption semantics. Regression tests cover interrupt-status preservation for both port-based and in-process server variants.

Next Guard

Do not use broad runCatching around blocking lifecycle APIs. If an API throws InterruptedException, use explicit catch and preserve interrupt status.

What These Mistakes Have in Common

The five cases look different, but they share the same pattern.

Mistake	Bad assumption	Actual guard
Empty numeric property	Empty value and 0 will be distinguished somehow	Format-specific round-trip test
Nightly external-service failures	Existing tests reflect current reality	Analyze Nightly failure as possible test drift
Memgraph timeout	A Testcontainers timeout must be a container problem	Separate query stack from cleanup stack
Vert.x cancellation	Route failure catch handles everything	Cancellation rethrow regression
gRPC interruption	runCatching can simplify lifecycle failure	Explicit interrupt preservation

The upside of an embarrassing bug is that the guard is usually clear afterward. It stops being an abstract design debate and becomes: “make this line fail earlier next time.”

How to Be a Little Less Wrong Next Time

I took three rules from these fixes.

First, do not infer value semantics from code types alone. "", null, and 0 are all different, especially in configuration files.

Second, do not guess the failure location. If Testcontainers appears in the failure, do not immediately fix the container. If the stack points to close(), look at cleanup.

Third, broad catch and runCatching are convenient but dangerous around lifecycle and coroutine boundaries. Cancellation and interruption are signals, not plain exceptions. If those signals are swallowed, the actual problem will show up somewhere much stranger later.

The checkDisabledTests release gate matters for the same reason. If an embarrassing mistake only lives in someone’s memory, it will repeat. It has to become a report, a test, a gate, or a lesson.

val checkDisabledTests by tasks.registering(DisabledTestReportTask::class) {
    description = "Scans JUnit disabled tests and fails known-bug skips without GitHub issue references."
    sourceRoot.set(layout.projectDirectory)
    sourceFiles.from(fileTree(layout.projectDirectory) {
        include("**/src/test/**/*.kt", "**/src/test/**/*.java")
    })
}

tasks.named("check") {
    dependsOn(checkDisabledTests)
}

Mistakes do not disappear. But they can be made easier to catch before release.

A good bug fix does not end at “fixed.” It should also mean “next time, this fails sooner.”