<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0"><channel><title><![CDATA[TapMap Engineering]]></title><description><![CDATA[TapMap Engineering]]></description><link>https://tapmap.hashnode.dev</link><image><url>https://cdn.hashnode.com/uploads/logos/6ae8235aa8f62719719ffa1a/b6b597cb-3d44-4011-b2da-954d58cf5d9f.png</url><title>TapMap Engineering</title><link>https://tapmap.hashnode.dev</link></image><generator>RSS for Node</generator><lastBuildDate>Sat, 07 Nov 2026 11:22:59 GMT</lastBuildDate><atom:link href="https://tapmap.hashnode.dev/rss.xml" rel="self" type="application/rss+xml"/><language><![CDATA[en]]></language><ttl>60</ttl><item><title><![CDATA[A map drawn by its users: how we built map and style marketplaces in TapMap]]></title><description><![CDATA[How TapMap lets people publish their own maps and map styles: layer versions, cache budgets, safe Mapbox styles and an API for AI agents.]]></description><link>https://tapmap.hashnode.dev/a-map-drawn-by-its-users-how-we-built-map-and-style-marketplaces-in-tapmap</link><guid isPermaLink="true">https://tapmap.hashnode.dev/a-map-drawn-by-its-users-how-we-built-map-and-style-marketplaces-in-tapmap</guid><category><![CDATA[Flutter]]></category><category><![CDATA[mapbox]]></category><category><![CDATA[architecture]]></category><category><![CDATA[PostgreSQL]]></category><dc:creator><![CDATA[Yurii Nazarenko]]></dc:creator><pubDate>Mon, 02 Nov 2026 06:16:45 GMT</pubDate><content:encoded><![CDATA[<p><strong>TL;DR.</strong> TapMap has two marketplaces. In the first one people publish maps: anyone collects their own set of places, and others switch it on as a layer over the main map. In the second one people publish styles: anyone (or their AI agent) repaints the map, adds their own icons and animations, and publishes the result. Below is how we got there, the mistakes we made and the decisions we took.</p>
<p><strong>Try the style editor:</strong> <a href="https://styles.tapmap.net/?utm_source=hashnode&amp;utm_medium=marketplaces">styles.tapmap.net</a></p>
<h2>The project in short</h2>
<p>TapMap is a social network built on maps. The question it answers is not "what's nearby" but "where should I go right now, and with whom". We started with Thailand.</p>
<p>The client is Flutter: Android, iOS and web from one codebase, with Riverpod for state. The map is Mapbox. The backend is Postgres with PostGIS.</p>
<p>Our main map is curated: places are collected and tagged by our own pipeline. The marketplaces are what people put on top of it.</p>
<h2>Part 1. The map marketplace</h2>
<h3>From moderation to personal maps</h3>
<p>At first we did what everyone does: a person adds a point, taps "Send for review", and a moderator decides whether it goes into the shared database. It scaled badly and it confused people: "I added my favourite coffee shop, where is it?"</p>
<p>So we flipped the model. A person's point belongs to their map and doesn't appear anywhere else. The shared database stays curated, and everyone gets their own maps:</p>
<ul>
<li>a default "My map" is created automatically on first sign-in;</li>
<li>you can create as many maps as you like, each with its own name, icon, colour and description;</li>
<li>you can invite friends as admins, and they add points too;</li>
<li>any map can be published to the marketplace;</li>
<li>someone else "downloads" it and turns it on as a layer, and there is no limit on how many layers are on at once.</li>
</ul>
<p>The data model turned out simple: a map, map members, map subscriptions, and a map reference on each point. No reference means the point is from the shared database; a reference means it's someone's collection. That gives one rule for all analytics: count the shared database separately, or other people's collections leak into the numbers.</p>
<h3>Permissions live in one place</h3>
<p>Access rules here are more than "this row is mine". They are a small piece of business logic: "only a friend can be made an admin", "you can only subscribe to a published map", "the default map can't be deleted". We put it into server-side functions instead of spreading it across declarative per-table policies. That way it's easier to read and harder to weaken by accident in the next change.</p>
<p>Ordering in the marketplace is simple for now: popular maps go first, and empty maps aren't shown at all. A shop window full of empty boxes is worse than an empty window.</p>
<h3>How a store chain directory nearly took the client down</h3>
<p>The client was designed for personal maps: a dozen or two favourite places. Then a map of a convenience store chain appeared in the marketplace, with more than sixteen thousand points. Three problems showed up at once.</p>
<ol>
<li>The whole layer was downloaded on every sync, even though the directory hadn't changed in weeks.</li>
<li>The big JSON was parsed on the main isolate. The stutter was visible.</li>
<li>The nastiest one: eviction. The main map lives entirely on the phone: the client downloads all points once and then asks the server only for changes. That cache has a ceiling, and map layers went into the same cache. The chain directory took up space, eviction threw out the oldest entries, which were points from the shared database. A subscriber to the directory silently lost thousands of cafés from the map, and sync didn't bring them back: from the server's point of view they hadn't changed.</li>
</ol>
<p>What we did.</p>
<p><strong>A version on every map.</strong> Any edit that changes what the client sees bumps the version number. The client first asks a cheap question, "what are the current versions of my maps?", and re-downloads only the maps whose number changed. A directory that hasn't changed isn't downloaded, isn't parsed and isn't written to the cache again.</p>
<p>The subtle part is deciding what counts as a change. Nightly enrichment runs touch points all the time: photos, ratings, attributes. If any touch bumped the version, the client would re-download the directory every night because of data that isn't even in the layer. So the version only grows when fields that actually end up in the layer change.</p>
<p>The version is a counter, not a "last modified" timestamp. Finding the maximum over sixteen thousand rows on every poll from every subscriber means paying CPU for the question "did anything change?". A counter answers it with a single read.</p>
<p><strong>Parsing in the background.</strong> Large responses are decoded in a separate isolate.</p>
<p><strong>Each kind of data gets its own budget.</strong> Map layers no longer share cache space with the main database. We bumped the snapshot format along the way, so everyone re-downloaded the old one in full once: phones already held snapshots "poisoned" by the directory.</p>
<p><strong>A big map is a directory, not a set of pins.</strong> A person's own pins are always visible: they're their places. But if sixteen thousand stores were shown the same way, they'd take every free spot before the cafés did. So a big map is treated as a directory: its points join the shared importance ranking and appear as you zoom in, and chain locations only get the space left over by regular places.</p>
<h3>The first frame comes from disk</h3>
<p>Each layer is stored on disk as a separate file together with its version. On a cold start, layers of enabled maps are loaded from disk before the network, so your own pins are there in the first frame and don't disappear offline.</p>
<p>Two more details that saved us debugging time:</p>
<ul>
<li>Syncs run strictly one at a time. Otherwise a retry after a failure and a scheduled pass would download the same layer twice and write the same file concurrently.</li>
<li>Account switches have a "generation". A response requested for the previous account no longer lands in the new account's cache after sign-out.</li>
</ul>
<pre><code class="language-dart">final generation = _generation;
bool stale() =&gt; generation != _generation || !(alive?.call() ?? true);
final run = _tail.then((_) async {
  if (stale()) return;
  // ...
});
_tail = run.then&lt;void&gt;((_) {}, onError: (Object _) {});
</code></pre>
<h3>Your choice survives a reinstall</h3>
<p>Which maps are on and which style is selected used to be remembered only by the device. A different phone, a messenger's in-app browser or a reinstall, and everything reset to defaults. Now the choice is stored in the account, and the newer record wins. Only the person's own actions are sent to the server, not settings restored at startup. Otherwise an old phone opened a week later would overwrite yesterday's choice made on the new one.</p>
<h3>Search finds maps too</h3>
<p>If you've downloaded a chain directory, searching "7 eleven" should find the map, not three hundred addresses. Your own maps and the most popular marketplace maps are matched by name right on the phone: from the start of any word, ignoring case, punctuation and spaces ("7 eleven" = "7-Eleven"). Picking a map in search turns it on and downloads it if needed.</p>
<h2>Part 2. The style marketplace</h2>
<h3>Before: four styles hard-coded in the app</h3>
<p>We had themed styles, Western, Neon, Terminal and Comic, each with its own place icons and animations. The app knew each of them by name in code. A fifth style meant a new app build.</p>
<p>We wanted anyone to be able to make a style, by hand in an editor or by handing the job to an AI agent. That meant a style had to become data, not code.</p>
<h3>A style pack: base style + manifest</h3>
<p>A style has two parts. The first is a regular Mapbox GL style: water, roads, parks, labels. The second is a manifest describing everything the app paints on top of that base:</p>
<pre><code class="language-json">{
  "name": {"ru": "Неоновый город", "en": "Neon city"},
  "base": "dark",
  "buildings": {
    "low": "#06131d", "mid": "#0a2734", "high": "#0f3d4d",
    "opacity": 0.8, "windows": "windows.png"
  },
  "icons": {
    "food":  {"image": "icons/food.png"},
    "clubs": {"frames": ["icons/clubs/f00.png", "icons/clubs/f01.png"], "fps": 12}
  }
}
</code></pre>
<ul>
<li><code>base</code> is a light or dark base. It decides how point labels are drawn.</li>
<li><code>buildings</code> are 3D buildings: wall colour by building height and a window texture.</li>
<li><code>icons</code> covers any icon the app draws: category pins, favourites, people's homes, event badges. Either a picture or a frame-by-frame animation. Anything missing from the manifest falls back to the app's own icon.</li>
</ul>
<h3>A style is code, even if it's JSON</h3>
<p>A Mapbox style contains URLs: where to get tiles, fonts and sprites. If a style pointing at someone else's server got into the marketplace, every phone using that style would start calling that server. From tile requests, that server would learn the approximate location of everyone looking at the map. For an app where people share their location with friends, that's unacceptable.</p>
<p>So marketplace styles are checked on the server, and third-party URLs aren't allowed. Besides security, the server makes sure a style isn't too heavy for a phone.</p>
<p>Full validation of the style itself (are the expressions valid, do the layers exist) happens in the browser editor. It renders the style with the same engine as the app and won't let you publish with errors. The split works out: the server is responsible for safety, the editor for correctness.</p>
<h3>An editor and an API for AI agents</h3>
<p>The editor lives at <a href="https://styles.tapmap.net/?utm_source=hashnode&amp;utm_medium=marketplaces">styles.tapmap.net</a>: repaint water and roads, change the label font, tune 3D buildings, replace icons with a picture or an animation.</p>
<p>The API does all of the same, and we designed it with AI agents in mind. A person gives an agent a key, and the agent works on their behalf. We learned a few things by watching agents make mistakes.</p>
<p><strong>Targeted edits instead of rewrites.</strong> It's awkward for an agent to rewrite a whole Mapbox style just to change the water colour. So there are operations: "paint layers matching a mask", "replace a colour everywhere", "remove a layer", "add a layer before a given one".</p>
<p><strong>An edit that touched nothing is an error, not silence.</strong> If an agent's layer mask misses, it finds out right away, not three steps later when the style "looks somehow wrong". Every response says how many layers were affected.</p>
<p><strong>A catalogue instead of guessing.</strong> A separate request returns every icon id with its size, the available fonts and the templates. The agent doesn't have to guess what exists.</p>
<p>In the end, "make it cyberpunk" turns into a few requests: create a draft from a template, repaint, upload icons, publish. The docs are open at <a href="https://styles.tapmap.net/docs">styles.tapmap.net/docs</a>.</p>
<h3>Publishing: versions are immutable</h3>
<p>Every publish creates a new version, and old ones are never touched. A phone that's in the middle of downloading version three will finish downloading it even if the author has already shipped version four. A new version is built on the side and becomes available all at once or not at all: there are no half-published versions.</p>
<p>All of a style's files are packed into one archive. An animated style is hundreds of small images, and a hundred requests over a mobile network are slower and flakier than one. The phone checks the archive checksum and won't install a pack that arrived corrupted. The format is the same as our built-in themed styles, so the app downloads both with the same loader.</p>
<p>People's styles live in a separate namespace in the app, so their names never clash with built-in packs and the loader knows right away where to fetch them from.</p>
<p>If a style is taken down from the marketplace, people who already installed it keep it. Taking someone's map away in the middle of a walk is bad UX.</p>
<h3>Why animation only plays on the selected pin</h3>
<p>We honestly wanted to animate every icon right on the map and tried the two obvious approaches. Both were bad:</p>
<ul>
<li>a separate layer per frame means copying the heaviest layer of the map (selection, collisions, labels) for every animation frame;</li>
<li>swapping the image on every frame flickers visibly and churns textures on the GPU.</li>
</ul>
<p>So on the map an animated icon shows its first frame. The animation plays on the selected pin, where the icon is drawn by a Flutter widget on top of the map. All frames are decoded in advance, otherwise the first loop of the animation flashes blank.</p>
<h3>Moderation without a queue</h3>
<p>Styles, like maps, are published immediately with no pre-moderation. Protection works after the fact: anyone can report a style right from the app, the team takes down violations, and the author can't republish a style that was taken down. The rules are in the terms of use, and the editor shows them as a checklist before publishing. The main one: no other people's logos or trademarks.</p>
<h3>Remixes</h3>
<p>Any published style can be copied and reworked, and the copy keeps a note of which style it came from. "Like theirs, only pink" takes one button, and the origin isn't lost.</p>
<h2>What we'd do differently</h2>
<ul>
<li><strong>Layer versions should have been there from day one.</strong> "Personal maps are small" holds right up to the first directory. A version counter and a file per map cost next to nothing; the rework with a migration of client snapshots cost noticeably more.</li>
<li><strong>Limits shouldn't be shared.</strong> A shared cache ceiling silently evicted points from the main database. We didn't notice anything until cafés disappeared from the map. Separate budgets for different kinds of data are much easier to debug.</li>
<li><strong>Anything that comes from users and runs on other people's devices is code.</strong> Even if it's "just a JSON style". Treat it accordingly.</li>
</ul>
<h2>What's next</h2>
<ul>
<li>sending changes to map layers instead of whole layers, once directories start changing often;</li>
<li>vector tiles instead of a list of points for the main map;</li>
<li>recommendations in the marketplaces: maps are sorted by popularity now, and we'd like to sort them by what fits a particular person too.</li>
</ul>
<p>You can try the style editor at <a href="https://styles.tapmap.net/?utm_source=hashnode&amp;utm_medium=marketplaces">styles.tapmap.net</a> (sign in with a TapMap account). The app itself is at <a href="https://tapmap.net/en/?utm_source=hashnode&amp;utm_medium=marketplaces">tapmap.net</a>.</p>
<p>If you've built something similar with custom Mapbox or MapLibre styles, especially icon animation without flicker, tell us in the comments how you solved it.</p>
]]></content:encoded></item><item><title><![CDATA[A whole country on the phone, without map clustering]]></title><description><![CDATA[How TapMap keeps 26,400 places on the phone with no map clustering: Flutter, Mapbox and PostGIS, with numbers for every step.]]></description><link>https://tapmap.hashnode.dev/a-whole-country-on-the-phone-without-map-clustering</link><guid isPermaLink="true">https://tapmap.hashnode.dev/a-whole-country-on-the-phone-without-map-clustering</guid><category><![CDATA[Flutter]]></category><category><![CDATA[mapbox]]></category><category><![CDATA[PostgreSQL]]></category><category><![CDATA[performance]]></category><dc:creator><![CDATA[Yurii Nazarenko]]></dc:creator><pubDate>Mon, 02 Nov 2026 05:56:32 GMT</pubDate><enclosure url="https://cdn.hashnode.com/uploads/covers/6ae8235aa8f62719719ffa1a/712d3871-0411-4e75-b94f-cee3d1176de7.jpg" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>We're building TapMap, a social network built on maps: instead of a feed, you get a map of places, people and their own maps. Stack: Flutter (Android, iOS and web from one codebase), Mapbox (<code>mapbox_maps_flutter</code> on phones, <code>mapbox-gl-js</code> in the browser) and self-hosted Supabase on Postgres 17 with PostGIS.</p>
<p>Today the database holds 26,400 published places in 6,185 cities, and the whole thing lives on the phone. There is no clustering, and there won't be. Below is how we got here, with the numbers for every step. Most of them are not flattering.</p>
<h2>Why no clustering</h2>
<p>A bubble that says "47" doesn't answer "where should I go?". You see that there's a lot of something somewhere, zoom in, the bubble splits into smaller bubbles, zoom in again. By the fourth zoom people close the app. We wanted specific places visible at every zoom level: the strongest ones from far away, all of them up close.</p>
<p>The product owner made the call with no compromise: no bubbles. Our job was to keep the map alive anyway.</p>
<h2>Step 0. Where we started: two megabytes per tap</h2>
<p>At 3,738 points, <code>map_points()</code> with no arguments returned <strong>2,093 KB</strong> on every launch, wherever the user was looking.</p>
<p>The client was worse. Every filter ran "string → <code>jsonDecode</code> → filter → <code>jsonEncode</code> → string", three filters in a row, plus a merge with the user's own points. That's <strong>five parses and four serialisations of two megabytes of JSON per filter tap</strong>, on the main isolate. If there was an event with a countdown in the next 24 hours, all of it repeated once a second.</p>
<p>The painful part: the function had <code>city_slug</code>, <code>bbox</code> and <code>max_points</code> parameters from day one. The client simply never passed them.</p>
<h2>Step 1. Parse once, query by viewport</h2>
<ul>
<li><strong>Parse once.</strong> JSON becomes a list of objects, all filters work on lists, and the string for the map is built once at the end.</li>
<li><strong>Query by viewport.</strong> When the camera stops, the screen reports its bounds. We pad the window by a screen width on each side and cap by zoom. A new request goes out only when the view leaves the loaded area or zoom crosses a threshold. Panning inside the padding costs zero network and zero parsing.</li>
</ul>
<p>Production numbers: the Phuket viewport returned <strong>432 points, 240 KB</strong>; a tight Patong window <strong>146 points, 81 KB</strong>. That's 9× and 26× less than 2,093 KB.</p>
<h2>Step 2. "The points disappeared"</h2>
<p>A day later we got a complaint: "most of the points on the map are gone". The first thinning caps were too greedy: 200 points below zoom 8, 400 below zoom 10.</p>
<ul>
<li><strong>All of Phuket in frame:</strong> 432 points uncapped, 400 capped. The 26 quietest beaches and nature spots vanished.</li>
<li><strong>Southern Thailand in one frame:</strong> 927 uncapped, 200 capped. Three quarters gone.</li>
</ul>
<p>We set a rule: <strong>a single city is never thinned.</strong> If someone is looking at a whole city, hiding points from them isn't optimisation, it's data loss. New caps: 400 up to zoom 7, 900 up to 9, 1,600 up to 11, everything after. Bangkok at "whole city in frame" gives 531 points under a 900 cap; Phuket shows all of its points. Thinning only starts when several cities are in frame.</p>
<h2>Step 3. Pins that lost silently</h2>
<p>Place pins in Mapbox didn't overlap (<code>icon-allow-overlap: false</code>). On collision the higher-rated pin stayed and the other one wasn't drawn at all. Below zoom 11 there was a layer of small dots for the "losers"; from zoom 11 it switched off. Result: at zooms 11–14 in dense areas most pins disappeared without a trace. On Kata Noi beach you could see one pin out of two; in Rawai, one out of four.</p>
<p>The fix: from zoom 11 pins may overlap, and only labels give way to each other. The threshold matches the zoom where labels turn on, on purpose.</p>
<h2>Step 4. "The map freezes for 1–2 seconds". It was the decor</h2>
<p>On a powerful phone the map would freeze for a second or two. On a weak one it didn't. That alone said it wasn't the points.</p>
<p>The cause was the scenery: trees, cars and 3D buildings. 700 ms after every camera stop, the decor asked Mapbox for roads, buildings and empty lots via <code>querySourceFeatures</code>. That method returns geometry <strong>from every loaded tile</strong>, not just the screen, and all of it gets decoded on the main isolate. Decor is disabled on weak phones, which is why they never froze.</p>
<p>Measured: Patong, zoom 15. Buildings alone took <strong>3,006 polygons in 2,206 ms</strong>; the whole pass took <strong>2,900 ms</strong>.</p>
<p>What we changed:</p>
<ol>
<li>Switched to <code>queryRenderedFeatures</code> for the screen rectangle and specific layers. For roads and empty lots we added two transparent "probe layers" with near-zero opacity. Exactly zero doesn't work: the engine doesn't render such a layer and returns nothing for it, so it's 0.02.</li>
<li>Decor is re-placed only when the camera moves a quarter of the screen or half a zoom step, not after every stop.</li>
<li>The delay before re-placing went from 0.7 s to 1.2 s, so it doesn't land between two finger movements.</li>
</ol>
<p>Same frame afterwards: <strong>71–184 ms</strong> instead of 2,900.</p>
<h2>Step 5. Cold start: that second wasn't the network</h2>
<p>Next complaint: "sometimes you wait a second before the points appear". Points were already cached in the background. The second was eaten by the queue of work before the first pin:</p>
<ul>
<li>8 glTF decor models and buildings waited <strong>before</strong> the "map ready" signal. Now decor comes in 700 ms after the points.</li>
<li>About 35 icons were decoded into bitmaps one by one just to read width and height. Now size is read from the PNG header and registration runs in parallel.</li>
<li>72 animation frame layers were added in 72 separate calls. Now it's one batch.</li>
</ul>
<p>Emulator, debug build: first frame <strong>+4.4 s → +1.6 s</strong>, points after first frame <strong>~6 s → ~1.1 s</strong>, image registration <strong>2,367 → 241 ms</strong>.</p>
<h2>Step 6. The whole database on the phone</h2>
<p>When places reached tens of thousands, we did what first looked reckless: we put the entire database on the phone.</p>
<ul>
<li>First launch downloads all of <code>map_points()</code>. For 26,400 points that's <strong>2.9 MB raw and 0.50 MB gzipped</strong>.</li>
<li>After that the app only asks for a delta: <code>map_points_changes(since)</code>.</li>
<li>The first frame is drawn from disk: camera, layers and ranks. Fresh data lands on top in the same places.</li>
</ul>
<h2>Step 7. Ranking decides who's visible, not bubbles</h2>
<p>Without clusters, something has to decide which point shows at which zoom. That's <code>PointsRanking</code>:</p>
<ul>
<li><strong>Importance</strong> comes from review count, rating and category weight.</li>
<li><strong>Below zoom 9</strong> the most important point wins in each grid cell.</li>
<li><strong>From zoom 9</strong> a point is visible if its pin has free space without collisions, like Google Maps and OpenMapTiles do it.</li>
<li><strong>From zoom 17</strong> everything is visible.</li>
<li><strong>Chains</strong> like 7-Eleven and Starbucks only get the space left over by regular places.</li>
</ul>
<p>The key fix came later: the selection is computed <strong>once per filter over the whole on-device database</strong>, not per camera window, and ties are broken by id. So every point has the same "appearance zoom" no matter how you move the map. Before, a point could blink on a two-pixel pan just because the window was recomputed and its cell neighbours changed.</p>
<h2>Where the ceiling is</h2>
<p>This can't last forever. There's a constant <code>PointsCache.cap = 40,000</code>: that's how many points still fit on a phone. It's the trigger to move to vector tiles (PMTiles), where thinning happens on the server at tile generation, not on the client. For now we're at 26 thousand, and the map opens faster than it used to at 3.7 thousand.</p>
<h2>Takeaways</h2>
<ol>
<li><strong>Measure before you fix.</strong> The freeze "caused by points" was decor. The "one second of loading" was image registration, not the network.</li>
<li><strong>The most dangerous bugs don't crash.</strong> A pin that silently lost to its neighbour leaves no log and no crash report. You only see it on a before/after screenshot.</li>
<li><strong>Thinning is a product decision, not a technical one.</strong> "Never hide points inside a single city" is a product rule, and once we had it, every number fell into place.</li>
</ol>
<p>If you're curious how the place cards are built (an LLM pipeline enriches them) or how we score a point's importance, ask in the comments and we'll write it up.</p>
<p><em>TapMap is a social network built on maps. Free on Android and on the web: <a href="https://tapmap.net/en/?utm_source=hashnode&amp;utm_medium=article&amp;utm_campaign=no_clustering">tapmap.net</a>.</em></p>
]]></content:encoded></item></channel></rss>