ChunkStream

Server-authoritative chunk-based data streaming system. The server manages all data in an in-memory spatial index and streams it to clients based on their chunk position. The client handles enter/leave detection locally using distance checks.

Concept

• Server holds all data, streams to clients, pushes deltas proactively
• Client caches data persistently, runs local enter/leave logic based on player distance
• Versioning — each chunk has a version counter; on re-entry only changed data is transferred
• Visibility — items can be restricted to specific characters, jobs, groups, or custom filters
• Bulk Loading — paginated loader for large datasets (10k+ items) with progress output
• Config Sync — chunk size and neighbor mode are configured on the server and automatically synced to clients via ready() / onReady()
• Multi-Stream — multiple independent streams per resource via create(stream_id)

Server API

---@type NssLibsChunkStreamServerApi
local stream_api = exports.nss_libs:getChunkStreamServerApi(GetCurrentResourceName())

---@type NssLibsChunkStreamServerStreamApi
local stream = stream_api.create('my_stream')

-- Configure before ready() (defaults apply if not set)
stream.setChunkSizeInMeters(100)
stream.setNeighborMode2D()
stream.setValidateClientPositionEnabled()

-- Signal readiness — config becomes available to clients
stream.ready()

-- Register tables after ready()
---@type NssLibsChunkStreamTableApi
local stream_table = stream.registerTable('markers')

stream_api.create(stream_id)

Creates a new stream instance for this resource.

Param	Type	Description
stream_id	string	Unique stream identifier (must match client-side)

Returns NssLibsChunkStreamServerStreamApi

---

stream.ready()

Signals that the stream is configured and ready. Config (chunk size, neighbor mode) becomes available to polling clients. Must be called before registerTable(). Config setters (setChunkSizeInMeters, setNeighborMode*) must be called before ready().

Returns NssLibsChunkStreamServerStreamApi — chainable

---

stream.registerTable(name)

Registers a data table and returns a table handle. Must be called after ready().

Param	Type	Description
name	string	Unique table name

Returns NssLibsChunkStreamTableApi

---

stream_table.addItem(item_id, x, y, z, radius, data, visibility?)

Adds an item to the spatial index.

Param	Type	Description
item_id	string	Unique item identifier
x	number	World X coordinate
y	number	World Y coordinate
z	number	World Z coordinate
radius	number	Visibility radius in meters
data	table	Arbitrary data payload (must be network-serializable — no functions, userdata, or circular references)
visibility	table|nil	Optional visibility rules (see below)

Returns NssLibsChunkStreamTableApi — chainable

---

stream_table.updateItem(item_id, data)

Updates an item's data without changing position or radius. The delta sent to clients contains only data (no position fields).

Param	Type	Description
item_id	string	Item identifier
data	table	New data payload

Returns NssLibsChunkStreamTableApi — chainable

---

stream_table.updateVisibility(item_id, visibility)

Updates an item's visibility rules without changing its data or position. Players who gain visibility receive an add delta (triggers onReceive). Players who lose visibility receive a remove delta (triggers onRemove). Players whose visibility is unchanged are not affected.

Param	Type	Description
item_id	string	Item identifier
visibility	table|nil	New visibility rules (see Visibility section), or nil for public

Returns NssLibsChunkStreamTableApi — chainable

---

stream_table.moveItem(item_id, x, y, z, radius?)

Moves an item to a new position. Optionally updates the radius. Internally generates up to three different delta types based on chunk overlap: update for chunks the item remains in (includes added_chunks/removed_chunks for client-side index reconciliation), add for chunks the item enters, and remove for chunks the item leaves. Each player receives exactly one delta (prioritized: update > add > remove).

Param	Type	Description
item_id	string	Item identifier
x	number	New X coordinate
y	number	New Y coordinate
z	number	New Z coordinate
radius	number|nil	New radius (keeps old if nil, must be positive)

Returns NssLibsChunkStreamTableApi — chainable

---

stream_table.removeItem(item_id)

Removes an item from the spatial index.

Param	Type	Description
item_id	string	Item identifier

Returns NssLibsChunkStreamTableApi — chainable

---

stream_table.getItem(item_id)

Returns the full item record or nil.

Param	Type	Description
item_id	string	Item identifier

Returns NssLibsChunkStreamItem|nil

Warning: Returns the internal item reference. Do not mutate the returned table directly — changes bypass version tracking and delta propagation. Use updateItem() or moveItem() to modify items.

Field	Type	Description
id	string	Item identifier
x	number	World X coordinate
y	number	World Y coordinate
z	number	World Z coordinate
radius	number	Visibility radius in meters
data	table	Arbitrary data payload
visibility	table|nil	Visibility rules
chunk_keys	table<string, boolean>	Set of chunk keys this item occupies

---

stream_table.getItemCount()

Returns the total number of items in this table.

Returns number

---

stream_table.setFilter(callback)

Sets a custom filter function called after visibility check.

Param	Type	Description
callback	function(item_id, data, source): boolean	Filter function

Returns NssLibsChunkStreamTableApi — chainable

---

stream_table.bulkLoad(options)

Paginated bulk loader for large datasets.

Param	Type	Description
options.totalItems	number	Total expected item count
options.pageSize	number	Items per page
options.loader	function(page, pageSize, tableHandle)	Loader callback
options.onComplete	function(total_loaded)|nil	Optional completion callback

Returns NssLibsChunkStreamTableApi — chainable

---

stream_table.clear()

Removes all items from this table. On the client, onRemove is called for every item the player is currently in radius of.

Returns NssLibsChunkStreamTableApi — chainable

Note: Do not call addItem directly on the table handle while a bulkLoad is in progress — delta pushes will be silently suppressed for those items.

Concurrency: If bulkLoad() is called while a bulk load is already in progress, the new options are queued and executed after the current load completes (FIFO order). Delta suppression remains active across the entire queue. Each call's onComplete fires independently when that individual load finishes.

---

Visibility

Items can have optional visibility rules. If set, the player must match at least one condition (OR logic):

stream_table.addItem('sheriff_office', x, y, z, 30.0, data, {
    char_ids = { 42, 43 },           -- Visible to these character IDs
    server_player_ids = { 1 },        -- Visible to these server source IDs
    jobs = { sheriff = 0 },           -- Hash format: delegated to char_data:hasOneOfTheJobs()
    groups = { 'admin' },             -- Visible to admins
})

-- Alternative: array format for jobs (simple name match, no grade check)
stream_table.addItem('depot', x, y, z, 20.0, data, {
    jobs = { 'sheriff', 'deputy' },   -- Array format: matches char_data.job
})

If visibility is nil, the item is visible to all players.

A custom filter (via setFilter) is evaluated after the visibility check. Both must pass.

---

Client API

---@type NssLibsChunkStreamClientApi
local stream_api = exports.nss_libs:getChunkStreamClientApi(GetCurrentResourceName())

---@type NssLibsChunkStreamClientStreamApi
local stream = stream_api.create('my_stream')

-- Client-specific settings (can be set before onReady)
stream.setPositionReportInterval(1000)
stream.setProximityCheckInterval(300)

-- Wait for server config, then register tables
stream.onReady(function(timeout)
    if timeout then return end

    ---@type NssLibsChunkStreamClientTableApi
    local stream_table = stream.registerTable('markers', {
        onReceive = function(item_id, data)
            -- Player entered item radius
        end,
        onUpdate = function(item_id, data)
            -- Item data changed while in radius (optional)
        end,
        onRemove = function(item_id)
            -- Player left item radius
        end,
    })
end)

stream_api.create(stream_id)

Creates a new client stream instance. The stream_id must match the server-side stream.

Param	Type	Description
stream_id	string	Stream identifier (must match server)

Returns NssLibsChunkStreamClientStreamApi

---

stream.onReady(callback, interval_ms?, timeout_ms?)

Polls the server until config is received, then fires the callback. Chunk size and neighbor mode are automatically applied from the server. registerTable() must be called inside this callback.

Param	Type	Description
callback	function(timeout: boolean)	Called when ready or on timeout
interval_ms	number|nil	Poll interval (default: 1000)
timeout_ms	number|nil	Timeout (default: 30000)

Returns NssLibsChunkStreamClientStreamApi — chainable

---

stream.registerTable(name, callbacks)

Registers a client-side table with enter/leave callbacks. Must be called inside onReady callback.

Param	Type	Description
name	string	Table name (must match server-side name)
callbacks.onReceive	function(item_id, data)	Required. Called when player enters item radius
callbacks.onUpdate	function(item_id, data)|nil	Optional. Called on data update while in radius. If omitted, onReceive is called instead. Position changes from moveItem update the cache silently — use getCached inside this callback to read the new position.
callbacks.onRemove	function(item_id)	Required. Called when player leaves item radius

Returns NssLibsChunkStreamClientTableApi

---

markers.getCached(item_id)

Returns cached data for an item the player is currently in radius of.

Returns table|nil

---

markers.getAllCached()

Returns all items the player is currently in radius of.

Returns table<string, table>

---

Configuration

Server Setters (stream.)

Defaults are defined in share/ChunkStreamDefaults.lua.

Pre-ready setters — must be called before ready() (synced to client):

Method	Default	Description
setChunkSizeInMeters(n)	50	Spatial grid cell size
setNeighborMode2D()	active	3x3 = 9 chunks
setNeighborMode3D()	—	3x3x3 = 27 chunks

Runtime setters — can be called at any time (server-only, not synced):

Method	Default	Description
setMaxChunkRequestsPerSecond(n)	20	Rate limit per player per second
setValidateClientPositionEnabled()	active	Server validates reported chunk key
setValidateClientPositionDisabled()	—	No position validation
setPositionValidationTolerance(n)	80	Tolerance in meters for validation

Client Setters (stream.)

Client-only settings. Can be called before onReady().

Method	Default	Description
setPositionReportInterval(ms)	1500	Chunk position check interval
setEnterDebounce(ms)	200	Delay before reporting chunk change. Rapid transitions within this window are merged (intermediate chunks skipped).
setProximityCheckInterval(ms)	500	Enter/leave check interval
setBatchProcessPerFrame(n)	50	Max items processed per frame

---

Dev notes

• The Chunk module is not modified. ChunkStream is fully independent.
• Multiple streams per resource are supported via create(stream_id). Each stream has its own spatial index, chunk versions, and player tracking.
• Config sync: The server signals readiness via ready(). Clients poll with ConfigPing events until the server responds with ConfigPong containing chunk size, decimal multiplier, and neighbor mode. Polling interval defaults to 1s, timeout to 30s.
• Lifecycle order: Server: config setters → ready() → registerTable(). Client: client-only setters → onReady(cb) → registerTable() inside callback.
• Chunk versions are never reset — they grow monotonically per chunk key.
• Client cache evicts stale chunks on each chunk change report — chunks outside the current neighbor set are removed from _chunk_data, _item_chunks, and _chunk_versions, preventing unbounded memory growth during long sessions. Items are also evicted via orphan detection during chunk refreshes and via remove/clear deltas. A reverse index (item → chunks) allows O(1) orphan detection instead of scanning all chunks.
• onChunkData writes cache entries (_chunk_data, _item_chunks) synchronously to keep the spatial index consistent. Only user-facing callbacks (onItemReceived) are deferred via _processBatch for frame yielding.
• clear() performs an O(1) wipe of items and chunks (no individual removeItem calls), then sends a clear delta to all connected players. clearAll() (used during destroy/resource stop) suppresses deltas.
• Deltas are batched per tick and sent as a single event per player, reducing network overhead when multiple items change simultaneously.
• moveItem deltas: The update delta from moveItem includes optional added_chunks/removed_chunks arrays so the client can reconcile its spatial index (_chunk_data, _item_chunks) atomically. Players with only new chunks get an add delta; players with only old chunks get a remove delta.
• bulkLoad queue: Concurrent bulkLoad() calls are queued (FIFO) rather than interleaved — _is_bulk_loading stays true until the entire queue is drained.
• Jobs visibility: Array format ({ 'sheriff' }) does a simple job == char_data.job comparison. Hash format ({ sheriff = 0 }) delegates to char_data:hasOneOfTheJobs(jobs) — the exact grade check behavior depends on the Character API implementation.
• updateVisibility: Sends add deltas to players who can now see the item and remove deltas to players who can no longer see it. Uses pushVisibilityChangeToActiveChunk on the stream which checks each active player's visibility individually. Idempotent on the client — duplicate adds overwrite cache, removes on missing items are no-ops.
• Chunk version pruning: pruneChunkVersionIfEmpty removes version entries for chunks that have no items in any table and no active players. Called after removeItem, moveItem (for vacated chunks), and clear. Prevents unbounded growth during long sessions with moving items.
• Clear delta chunk versions: pushClearDelta includes a chunk_versions map with all affected chunk versions. The client applies all versions on clear, preventing unnecessary resync on the next chunk change.