## Breaking Change
IIO API redesigned from manual pull+if-forest to callback dispatch.
All modules must update their subscribe() calls to pass handlers.
### Before (OLD API)
```cpp
io->subscribe("input:mouse");
void process(...) {
while (io->hasMessages()) {
auto msg = io->pullMessage();
if (msg.topic == "input:mouse") {
handleMouse(msg);
} else if (msg.topic == "input:keyboard") {
handleKeyboard(msg);
}
}
}
```
### After (NEW API)
```cpp
io->subscribe("input:mouse", [this](const Message& msg) {
handleMouse(msg);
});
void process(...) {
while (io->hasMessages()) {
io->pullAndDispatch(); // Callbacks invoked automatically
}
}
```
## Changes
**Core API (include/grove/IIO.h)**
- Added: `using MessageHandler = std::function<void(const Message&)>`
- Changed: `subscribe()` now requires `MessageHandler` callback parameter
- Changed: `subscribeLowFreq()` now requires `MessageHandler` callback
- Removed: `pullMessage()`
- Added: `pullAndDispatch()` - pulls and auto-dispatches to handlers
**Implementation (src/IntraIO.cpp)**
- Store callbacks in `Subscription.handler`
- `pullAndDispatch()` matches topic against ALL subscriptions (not just first)
- Fixed: Regex pattern compilation supports both wildcards (*) and regex (.*)
- Performance: ~1000 msg/s throughput (unchanged from before)
**Files Updated**
- 31 test/module files migrated to callback API (via parallel agents)
- 8 documentation files updated (DEVELOPER_GUIDE, USER_GUIDE, module READMEs)
## Bugs Fixed During Migration
1. **pullAndDispatch() early return bug**: Was only calling FIRST matching handler
- Fix: Loop through ALL subscriptions, invoke all matching handlers
2. **Regex pattern compilation bug**: Pattern "player:.*" failed to match
- Fix: Detect ".*" in pattern → use as regex, otherwise escape and convert wildcards
## Testing
✅ test_11_io_system: PASSED (IIO pub/sub, pattern matching, batching)
✅ test_threaded_module_system: 6/6 PASSED
✅ test_threaded_stress: 5/5 PASSED (50 modules, 100x reload, concurrent ops)
✅ test_12_datanode: PASSED
✅ 10 TopicTree scenarios: 10/10 PASSED
✅ benchmark_e2e: ~1000 msg/s throughput
Total: 23+ tests passing
## Performance Impact
No performance regression from callback dispatch:
- IIO throughput: ~1000 msg/s (same as before)
- ThreadedModuleSystem: Speedup ~1.0x (barrier pattern expected)
## Migration Guide
For all modules using IIO:
1. Update subscribe() calls to include handler lambda
2. Replace pullMessage() loops with pullAndDispatch()
3. Move topic-specific logic from if-forest into callbacks
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
153 lines
4.9 KiB
C++
153 lines
4.9 KiB
C++
#pragma once
|
|
|
|
#include <string>
|
|
#include <vector>
|
|
#include <functional>
|
|
#include <memory>
|
|
#include <cstdint>
|
|
#include "IDataNode.h"
|
|
|
|
namespace grove {
|
|
|
|
enum class IOType {
|
|
INTRA = 0, // Same process
|
|
LOCAL = 1, // Same machine
|
|
NETWORK = 2 // TCP/WebSocket
|
|
};
|
|
|
|
struct SubscriptionConfig {
|
|
bool replaceable = false; // Replace vs accumulate for low-freq
|
|
int batchInterval = 30000; // ms for low-freq batching
|
|
int maxBatchSize = 100; // Max messages per batch
|
|
bool compress = false; // Compress batched data
|
|
};
|
|
|
|
struct Message {
|
|
std::string topic;
|
|
std::unique_ptr<IDataNode> data;
|
|
uint64_t timestamp;
|
|
|
|
// Default constructor
|
|
Message() = default;
|
|
|
|
// Move constructor and assignment (unique_ptr is move-only)
|
|
Message(Message&&) = default;
|
|
Message& operator=(Message&&) = default;
|
|
|
|
// Delete copy (unique_ptr cannot be copied)
|
|
Message(const Message&) = delete;
|
|
Message& operator=(const Message&) = delete;
|
|
};
|
|
|
|
struct IOHealth {
|
|
int queueSize;
|
|
int maxQueueSize;
|
|
bool dropping = false; // Started dropping messages?
|
|
float averageProcessingRate; // Messages/second processed by module
|
|
int droppedMessageCount = 0; // Total dropped since last check
|
|
};
|
|
|
|
/**
|
|
* @brief Message handler callback type
|
|
*
|
|
* Callback invoked when a message matching the subscribed pattern is pulled.
|
|
* Module implements this to handle specific message types without if-forest dispatch.
|
|
*/
|
|
using MessageHandler = std::function<void(const Message&)>;
|
|
|
|
/**
|
|
* @brief Pub/Sub communication interface with pull-based callback dispatch
|
|
*
|
|
* Pull-based pub/sub system with automatic message dispatch to registered handlers.
|
|
* Modules subscribe with callbacks, then pull messages - dispatch is automatic.
|
|
*
|
|
* Design:
|
|
* - Modules retain control over WHEN to process (pull-based)
|
|
* - No if-forest dispatch (callbacks registered at subscription)
|
|
* - Thread-safe for multi-threaded module systems
|
|
*
|
|
* Features:
|
|
* - Topic patterns with wildcards (e.g., "player:*", "economy:*")
|
|
* - Low-frequency subscriptions for bandwidth optimization
|
|
* - Automatic callback dispatch on pull
|
|
* - Engine health monitoring for backpressure management
|
|
*/
|
|
class IIO {
|
|
public:
|
|
virtual ~IIO() = default;
|
|
|
|
/**
|
|
* @brief Publish message to a topic
|
|
* @param topic Topic name (e.g., "player:123", "economy:prices")
|
|
* @param message Message data
|
|
*/
|
|
virtual void publish(const std::string& topic, std::unique_ptr<IDataNode> message) = 0;
|
|
|
|
/**
|
|
* @brief Subscribe to topic pattern with callback handler (high-frequency)
|
|
* @param topicPattern Topic pattern with wildcards (e.g., "player:*")
|
|
* @param handler Callback invoked when matching message is pulled
|
|
* @param config Optional subscription configuration
|
|
*
|
|
* Example:
|
|
* io->subscribe("input:mouse", [this](const Message& msg) {
|
|
* handleMouseInput(msg);
|
|
* });
|
|
*/
|
|
virtual void subscribe(
|
|
const std::string& topicPattern,
|
|
MessageHandler handler,
|
|
const SubscriptionConfig& config = {}
|
|
) = 0;
|
|
|
|
/**
|
|
* @brief Subscribe to topic pattern with callback (low-frequency batched)
|
|
* @param topicPattern Topic pattern with wildcards
|
|
* @param handler Callback invoked when matching message is pulled
|
|
* @param config Subscription configuration (batchInterval, etc.)
|
|
*
|
|
* Example:
|
|
* io->subscribeLowFreq("analytics:*", [this](const Message& msg) {
|
|
* processBatchedAnalytics(msg);
|
|
* }, {.batchInterval = 5000});
|
|
*/
|
|
virtual void subscribeLowFreq(
|
|
const std::string& topicPattern,
|
|
MessageHandler handler,
|
|
const SubscriptionConfig& config = {}
|
|
) = 0;
|
|
|
|
/**
|
|
* @brief Get count of pending messages
|
|
* @return Number of messages waiting to be pulled
|
|
*/
|
|
virtual int hasMessages() const = 0;
|
|
|
|
/**
|
|
* @brief Pull and auto-dispatch one message to registered handler
|
|
* @throws std::runtime_error if no messages available
|
|
*
|
|
* Pulls oldest message from queue and invokes the callback registered
|
|
* during subscribe(). Message is consumed (removed from queue).
|
|
*
|
|
* Example usage:
|
|
* while (io->hasMessages() > 0) {
|
|
* io->pullAndDispatch(); // Callbacks invoked automatically
|
|
* }
|
|
*/
|
|
virtual void pullAndDispatch() = 0;
|
|
|
|
/**
|
|
* @brief Get IO health status for Engine monitoring
|
|
* @return Health metrics including queue size, drop status, processing rate
|
|
*/
|
|
virtual IOHealth getHealth() const = 0;
|
|
|
|
/**
|
|
* @brief Get IO type identifier
|
|
* @return IO type enum value for identification
|
|
*/
|
|
virtual IOType getType() const = 0;
|
|
};
|
|
|
|
} // namespace grove
|