StillHammer/GroveEngine
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: Add comprehensive hot-reload test suite with 3 integration scenarios

Browse Source 
This commit implements a complete test infrastructure for validating
hot-reload stability and robustness across multiple scenarios.

## New Test Infrastructure

### Test Helpers (tests/helpers/)
- TestMetrics: FPS, memory, reload time tracking with statistics
- TestReporter: Assertion tracking and formatted test reports
- SystemUtils: Memory usage monitoring via /proc/self/status
- TestAssertions: Macro-based assertion framework

### Test Modules
- TankModule: Realistic module with 50 tanks for production testing
- ChaosModule: Crash-injection module for robustness validation
- StressModule: Lightweight module for long-duration stability tests

## Integration Test Scenarios

### Scenario 1: Production Hot-Reload (test_01_production_hotreload.cpp)
✅ PASSED - End-to-end hot-reload validation
- 30 seconds simulation (1800 frames @ 60 FPS)
- TankModule with 50 tanks, realistic state
- Source modification (v1.0 → v2.0), recompilation, reload
- State preservation: positions, velocities, frameCount
- Metrics: ~163ms reload time, 0.88MB memory growth

### Scenario 2: Chaos Monkey (test_02_chaos_monkey.cpp)
✅ PASSED - Extreme robustness testing
- 150+ random crashes per run (5% crash probability per frame)
- 5 crash types: runtime_error, logic_error, out_of_range, domain_error, state corruption
- 100% recovery rate via automatic hot-reload
- Corrupted state detection and rejection
- Random seed for unpredictable crash patterns
- Proof of real reload: temporary files in /tmp/grove_module_*.so

### Scenario 3: Stress Test (test_03_stress_test.cpp)
✅ PASSED - Long-duration stability validation
- 10 minutes simulation (36000 frames @ 60 FPS)
- 120 hot-reloads (every 5 seconds)
- 100% reload success rate (120/120)
- Memory growth: 2 MB (threshold: 50 MB)
- Avg reload time: 160ms (threshold: 500ms)
- No memory leaks, no file descriptor leaks

## Core Engine Enhancements

### ModuleLoader (src/ModuleLoader.cpp)
- Temporary file copy to /tmp/ for Linux dlopen cache bypass
- Robust reload() method: getState() → unload() → load() → setState()
- Automatic cleanup of temporary files
- Comprehensive error handling and logging

### DebugEngine (src/DebugEngine.cpp)
- Automatic recovery in processModuleSystems()
- Exception catching → logging → module reload → continue
- Module state dump utilities for debugging

### SequentialModuleSystem (src/SequentialModuleSystem.cpp)
- extractModule() for safe module extraction
- registerModule() for module re-registration
- Enhanced processModules() with error handling

## Build System
- CMake configuration for test infrastructure
- Shared library compilation for test modules (.so)
- CTest integration for all scenarios
- PIC flag management for spdlog compatibility

## Documentation (planTI/)
- Complete test architecture documentation
- Detailed scenario specifications with success criteria
- Global test plan and validation thresholds

## Validation Results
All 3 integration scenarios pass successfully:
- Production hot-reload: State preservation validated
- Chaos Monkey: 100% recovery from 150+ crashes
- Stress Test: Stable over 120 reloads, minimal memory growth

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
StillHammer 2025-11-13 22:13:07 +08:00
parent d9a76395f5
commit d8c5f93429
37 changed files with 6202 additions and 37 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
CMakeLists.txt
View File

@ -17,6 +17,7 @@ FetchContent_Declare(
FetchContent_MakeAvailable(nlohmann_json) FetchContent_MakeAvailable(nlohmann_json)
# spdlog for logging # spdlog for logging
set(CMAKE_POSITION_INDEPENDENT_CODE ON) # Force PIC for all targets
FetchContent_Declare( FetchContent_Declare(
spdlog spdlog
GIT_REPOSITORY https://github.com/gabime/spdlog.git GIT_REPOSITORY https://github.com/gabime/spdlog.git

17
include/grove/DebugEngine.h
View File

@ -111,6 +111,23 @@ public:
* @brief Get list of all registered module names * @brief Get list of all registered module names
*/ */
std::vector<std::string> getModuleNames() const { return moduleNames; } std::vector<std::string> getModuleNames() const { return moduleNames; }
/**
* @brief Dump the current state of a specific module to logs
* @param name Module identifier
*
* Retrieves the module's state via getState() and pretty-prints it
* as formatted JSON in the logs. Useful for debugging and inspection.
*/
void dumpModuleState(const std::string& name);
/**
* @brief Dump the state of all registered modules to logs
*
* Iterates through all modules and dumps their state.
* Useful for comprehensive system state snapshots.
*/
void dumpAllModulesState();
}; };
} // namespace grove } // namespace grove

18
include/grove/IModule.h
View File

@ -108,6 +108,24 @@ public:
* @return Module type as string (e.g., "tank", "economy", "production") * @return Module type as string (e.g., "tank", "economy", "production")
*/ */
virtual std::string getType() const = 0; virtual std::string getType() const = 0;
/**
* @brief Check if module is idle (no processing in progress)
* @return True if module has no active processing and can be safely hot-reloaded
*
* Used by hot-reload system to ensure safe reload timing.
* A module is considered idle when:
* - No synchronous processing in progress
* - Not waiting for critical state updates
* - Safe to extract state via getState()
*
* Note: Async tasks scheduled via ITaskScheduler are tracked separately
* by the module system and don't affect idle status.
*
* Default implementation should return true unless module explicitly
* tracks long-running synchronous operations.
*/
virtual bool isIdle() const = 0;
}; };
} // namespace grove } // namespace grove

11
include/grove/IModuleSystem.h
View File

@ -87,6 +87,17 @@ public:
* @return Module system type enum value for identification * @return Module system type enum value for identification
*/ */
virtual ModuleSystemType getType() const = 0; virtual ModuleSystemType getType() const = 0;
/**
* @brief Get count of pending async tasks for a specific module
* @param moduleName Name of the module to check
* @return Number of tasks still pending (scheduled or executing)
*
* Used by hot-reload system to ensure all async operations complete
* before module replacement. Returns 0 if module has no pending tasks
* or module name is unknown.
*/
virtual int getPendingTaskCount(const std::string& moduleName) const = 0;
}; };
} // namespace grove } // namespace grove

21
include/grove/ModuleLoader.h
View File

@ -26,6 +26,7 @@ private:
void* libraryHandle = nullptr; void* libraryHandle = nullptr;
std::string libraryPath; std::string libraryPath;
std::string moduleName; std::string moduleName;
std::string tempLibraryPath; // Temp copy path for hot-reload cache bypass
// Factory function signature: IModule* createModule() // Factory function signature: IModule* createModule()
using CreateModuleFunc = IModule* (*)(); using CreateModuleFunc = IModule* (*)();
@ -45,10 +46,11 @@ public:
* @brief Load a module from .so file * @brief Load a module from .so file
* @param path Path to .so file * @param path Path to .so file
* @param moduleName Name for logging/identification * @param moduleName Name for logging/identification
* @param isReload If true, use temp copy to bypass dlopen cache (default: false)
* @return Unique pointer to loaded module * @return Unique pointer to loaded module
* @throws std::runtime_error if loading fails * @throws std::runtime_error if loading fails
*/ */
std::unique_ptr<IModule> load(const std::string& path, const std::string& name); std::unique_ptr<IModule> load(const std::string& path, const std::string& name, bool isReload = false);
/** /**
* @brief Unload currently loaded module * @brief Unload currently loaded module
@ -84,6 +86,23 @@ public:
* @brief Get name of currently loaded module * @brief Get name of currently loaded module
*/ */
const std::string& getModuleName() const { return moduleName; } const std::string& getModuleName() const { return moduleName; }
/**
* @brief Wait for module to reach clean state (idle + no pending tasks)
* @param module Module to wait for
* @param moduleSystem Module system to check for pending tasks
* @param timeoutSeconds Maximum time to wait in seconds (default: 5.0)
* @return True if clean state reached, false if timeout
*
* Used by hot-reload to ensure safe reload timing. Waits until:
* - module->isIdle() returns true
* - moduleSystem->getPendingTaskCount() returns 0
*/
bool waitForCleanState(
IModule* module,
class IModuleSystem* moduleSystem,
float timeoutSeconds = 5.0f
);
}; };
} // namespace grove } // namespace grove

1
include/grove/SequentialModuleSystem.h
View File

@ -67,6 +67,7 @@ public:
void setIOLayer(std::unique_ptr<IIO> ioLayer) override; void setIOLayer(std::unique_ptr<IIO> ioLayer) override;
std::unique_ptr<IDataNode> queryModule(const std::string& name, const IDataNode& input) override; std::unique_ptr<IDataNode> queryModule(const std::string& name, const IDataNode& input) override;
ModuleSystemType getType() const override; ModuleSystemType getType() const override;
int getPendingTaskCount(const std::string& moduleName) const override;
// Hot-reload support // Hot-reload support
std::unique_ptr<IModule> extractModule(); std::unique_ptr<IModule> extractModule();

634
planTI/architecture_tests.md Normal file
View File

@ -0,0 +1,634 @@
# Architecture des Tests - Helpers & Infrastructure
Ce document détaille l'architecture commune à tous les tests d'intégration.
---
## 📁 Structure des Fichiers
```
tests/
├─ integration/
│ ├─ test_01_production_hotreload.cpp
│ ├─ test_02_chaos_monkey.cpp
│ ├─ test_03_stress_test.cpp
│ ├─ test_04_race_condition.cpp
│ └─ test_05_multimodule.cpp
│
├─ modules/
│ ├─ TankModule.h/.cpp # Module réaliste avec state complexe
│ ├─ ProductionModule.h/.cpp # Auto-spawn entities
│ ├─ MapModule.h/.cpp # Grille 2D
│ ├─ ChaosModule.h/.cpp # Génère failures aléatoires
│ └─ HeavyStateModule.h/.cpp # State 100MB (Phase 3)
│
└─ helpers/
├─ TestMetrics.h/.cpp # Collecte métriques (memory, FPS, etc.)
├─ TestAssertions.h # Macros d'assertions
├─ TestReporter.h/.cpp # Génération rapports pass/fail
├─ ResourceMonitor.h/.cpp # Monitoring CPU, FD, etc.
├─ AutoCompiler.h/.cpp # Compilation automatique
└─ SystemUtils.h/.cpp # Utilitaires système (memory, FD, CPU)
```
---
## 🔧 Helpers Détaillés
### 1. TestMetrics
**Fichier**: `tests/helpers/TestMetrics.h` et `TestMetrics.cpp`
**Responsabilité**: Collecter toutes les métriques durant l'exécution des tests.
```cpp
// TestMetrics.h
#pragma once
#include <vector>
#include <algorithm>
#include <numeric>
#include <cmath>
class TestMetrics {
public:
// Enregistrement
void recordFPS(float fps);
void recordMemoryUsage(size_t bytes);
void recordReloadTime(float ms);
void recordCrash(const std::string& reason);
// Getters - FPS
float getFPSMin() const;
float getFPSMax() const;
float getFPSAvg() const;
float getFPSStdDev() const;
float getFPSMinLast60s() const; // Pour stress test
float getFPSAvgLast60s() const;
// Getters - Memory
size_t getMemoryInitial() const;
size_t getMemoryFinal() const;
size_t getMemoryPeak() const;
size_t getMemoryGrowth() const;
// Getters - Reload
float getReloadTimeAvg() const;
float getReloadTimeMin() const;
float getReloadTimeMax() const;
float getReloadTimeP99() const; // Percentile 99
int getReloadCount() const;
// Getters - Crashes
int getCrashCount() const;
const std::vector<std::string>& getCrashReasons() const;
// Rapport
void printReport() const;
private:
std::vector<float> fpsValues;
std::vector<size_t> memoryValues;
std::vector<float> reloadTimes;
std::vector<std::string> crashReasons;
size_t initialMemory = 0;
bool hasInitialMemory = false;
};
```
```cpp
// TestMetrics.cpp
#include "TestMetrics.h"
#include <iostream>
#include <iomanip>
void TestMetrics::recordFPS(float fps) {
fpsValues.push_back(fps);
}
void TestMetrics::recordMemoryUsage(size_t bytes) {
if (!hasInitialMemory) {
initialMemory = bytes;
hasInitialMemory = true;
}
memoryValues.push_back(bytes);
}
void TestMetrics::recordReloadTime(float ms) {
reloadTimes.push_back(ms);
}
void TestMetrics::recordCrash(const std::string& reason) {
crashReasons.push_back(reason);
}
float TestMetrics::getFPSMin() const {
if (fpsValues.empty()) return 0.0f;
return *std::min_element(fpsValues.begin(), fpsValues.end());
}
float TestMetrics::getFPSMax() const {
if (fpsValues.empty()) return 0.0f;
return *std::max_element(fpsValues.begin(), fpsValues.end());
}
float TestMetrics::getFPSAvg() const {
if (fpsValues.empty()) return 0.0f;
return std::accumulate(fpsValues.begin(), fpsValues.end(), 0.0f) / fpsValues.size();
}
float TestMetrics::getFPSStdDev() const {
if (fpsValues.empty()) return 0.0f;
float avg = getFPSAvg();
float variance = 0.0f;
for (float fps : fpsValues) {
variance += std::pow(fps - avg, 2);
}
return std::sqrt(variance / fpsValues.size());
}
size_t TestMetrics::getMemoryGrowth() const {
if (memoryValues.empty()) return 0;
return memoryValues.back() - initialMemory;
}
size_t TestMetrics::getMemoryPeak() const {
if (memoryValues.empty()) return 0;
return *std::max_element(memoryValues.begin(), memoryValues.end());
}
float TestMetrics::getReloadTimeAvg() const {
if (reloadTimes.empty()) return 0.0f;
return std::accumulate(reloadTimes.begin(), reloadTimes.end(), 0.0f) / reloadTimes.size();
}
float TestMetrics::getReloadTimeP99() const {
if (reloadTimes.empty()) return 0.0f;
auto sorted = reloadTimes;
std::sort(sorted.begin(), sorted.end());
size_t p99Index = static_cast<size_t>(sorted.size() * 0.99);
return sorted[p99Index];
}
void TestMetrics::printReport() const {
std::cout << "╔══════════════════════════════════════════════════════════════\n";
std::cout << "║ METRICS REPORT\n";
std::cout << "╠══════════════════════════════════════════════════════════════\n";
if (!fpsValues.empty()) {
std::cout << "║ FPS:\n";
std::cout << "║ Min: " << std::setw(8) << getFPSMin() << "\n";
std::cout << "║ Avg: " << std::setw(8) << getFPSAvg() << "\n";
std::cout << "║ Max: " << std::setw(8) << getFPSMax() << "\n";
std::cout << "║ Std Dev: " << std::setw(8) << getFPSStdDev() << "\n";
}
if (!memoryValues.empty()) {
std::cout << "║ Memory:\n";
std::cout << "║ Initial: " << std::setw(8) << (initialMemory / 1024.0f / 1024.0f) << " MB\n";
std::cout << "║ Final: " << std::setw(8) << (memoryValues.back() / 1024.0f / 1024.0f) << " MB\n";
std::cout << "║ Peak: " << std::setw(8) << (getMemoryPeak() / 1024.0f / 1024.0f) << " MB\n";
std::cout << "║ Growth: " << std::setw(8) << (getMemoryGrowth() / 1024.0f / 1024.0f) << " MB\n";
}
if (!reloadTimes.empty()) {
std::cout << "║ Reload Times:\n";
std::cout << "║ Count: " << std::setw(8) << reloadTimes.size() << "\n";
std::cout << "║ Avg: " << std::setw(8) << getReloadTimeAvg() << " ms\n";
std::cout << "║ Min: " << std::setw(8) << getReloadTimeMin() << " ms\n";
std::cout << "║ Max: " << std::setw(8) << getReloadTimeMax() << " ms\n";
std::cout << "║ P99: " << std::setw(8) << getReloadTimeP99() << " ms\n";
}
if (!crashReasons.empty()) {
std::cout << "║ Crashes: " << crashReasons.size() << "\n";
for (const auto& reason : crashReasons) {
std::cout << "║ - " << reason << "\n";
}
}
std::cout << "╚══════════════════════════════════════════════════════════════\n";
}
```
---
### 2. TestAssertions
**Fichier**: `tests/helpers/TestAssertions.h` (header-only)
**Responsabilité**: Macros d'assertions pour tests.
```cpp
// TestAssertions.h
#pragma once
#include <iostream>
#include <cstdlib>
#include <cmath>
// Couleurs pour output
#define COLOR_RED "\033[31m"
#define COLOR_GREEN "\033[32m"
#define COLOR_RESET "\033[0m"
#define ASSERT_TRUE(condition, message) \
do { \
if (!(condition)) { \
std::cerr << COLOR_RED << "❌ ASSERTION FAILED: " << message << COLOR_RESET << "\n"; \
std::cerr << " At: " << __FILE__ << ":" << __LINE__ << "\n"; \
std::exit(1); \
} \
} while(0)
#define ASSERT_FALSE(condition, message) \
ASSERT_TRUE(!(condition), message)
#define ASSERT_EQ(actual, expected, message) \
do { \
if ((actual) != (expected)) { \
std::cerr << COLOR_RED << "❌ ASSERTION FAILED: " << message << COLOR_RESET << "\n"; \
std::cerr << " Expected: " << (expected) << "\n"; \
std::cerr << " Actual: " << (actual) << "\n"; \
std::cerr << " At: " << __FILE__ << ":" << __LINE__ << "\n"; \
std::exit(1); \
} \
} while(0)
#define ASSERT_NE(actual, expected, message) \
do { \
if ((actual) == (expected)) { \
std::cerr << COLOR_RED << "❌ ASSERTION FAILED: " << message << COLOR_RESET << "\n"; \
std::cerr << " Should not equal: " << (expected) << "\n"; \
std::cerr << " But got: " << (actual) << "\n"; \
std::cerr << " At: " << __FILE__ << ":" << __LINE__ << "\n"; \
std::exit(1); \
} \
} while(0)
#define ASSERT_LT(value, max, message) \
do { \
if ((value) >= (max)) { \
std::cerr << COLOR_RED << "❌ ASSERTION FAILED: " << message << COLOR_RESET << "\n"; \
std::cerr << " Expected: < " << (max) << "\n"; \
std::cerr << " Actual: " << (value) << "\n"; \
std::cerr << " At: " << __FILE__ << ":" << __LINE__ << "\n"; \
std::exit(1); \
} \
} while(0)
#define ASSERT_GT(value, min, message) \
do { \
if ((value) <= (min)) { \
std::cerr << COLOR_RED << "❌ ASSERTION FAILED: " << message << COLOR_RESET << "\n"; \
std::cerr << " Expected: > " << (min) << "\n"; \
std::cerr << " Actual: " << (value) << "\n"; \
std::cerr << " At: " << __FILE__ << ":" << __LINE__ << "\n"; \
std::exit(1); \
} \
} while(0)
#define ASSERT_WITHIN(actual, expected, tolerance, message) \
do { \
auto diff = std::abs((actual) - (expected)); \
if (diff > (tolerance)) { \
std::cerr << COLOR_RED << "❌ ASSERTION FAILED: " << message << COLOR_RESET << "\n"; \
std::cerr << " Expected: " << (expected) << " ± " << (tolerance) << "\n"; \
std::cerr << " Actual: " << (actual) << " (diff: " << diff << ")\n"; \
std::cerr << " At: " << __FILE__ << ":" << __LINE__ << "\n"; \
std::exit(1); \
} \
} while(0)
```
---
### 3. TestReporter
**Fichier**: `tests/helpers/TestReporter.h` et `TestReporter.cpp`
**Responsabilité**: Générer rapport final pass/fail.
```cpp
// TestReporter.h
#pragma once
#include <string>
#include <map>
#include <vector>
class TestReporter {
public:
explicit TestReporter(const std::string& scenarioName);
void addMetric(const std::string& name, float value);
void addAssertion(const std::string& name, bool passed);
void printFinalReport() const;
int getExitCode() const; // 0 = pass, 1 = fail
private:
std::string scenarioName;
std::map<std::string, float> metrics;
std::vector<std::pair<std::string, bool>> assertions;
};
```
```cpp
// TestReporter.cpp
#include "TestReporter.h"
#include <iostream>
TestReporter::TestReporter(const std::string& name) : scenarioName(name) {}
void TestReporter::addMetric(const std::string& name, float value) {
metrics[name] = value;
}
void TestReporter::addAssertion(const std::string& name, bool passed) {
assertions.push_back({name, passed});
}
void TestReporter::printFinalReport() const {
std::cout << "\n";
std::cout << "════════════════════════════════════════════════════════════════\n";
std::cout << "FINAL REPORT: " << scenarioName << "\n";
std::cout << "════════════════════════════════════════════════════════════════\n\n";
// Metrics
if (!metrics.empty()) {
std::cout << "Metrics:\n";
for (const auto& [name, value] : metrics) {
std::cout << " " << name << ": " << value << "\n";
}
std::cout << "\n";
}
// Assertions
if (!assertions.empty()) {
std::cout << "Assertions:\n";
bool allPassed = true;
for (const auto& [name, passed] : assertions) {
std::cout << " " << (passed ? "✓" : "✗") << " " << name << "\n";
if (!passed) allPassed = false;
}
std::cout << "\n";
if (allPassed) {
std::cout << "Result: ✅ PASSED\n";
} else {
std::cout << "Result: ❌ FAILED\n";
}
}
std::cout << "════════════════════════════════════════════════════════════════\n";
}
int TestReporter::getExitCode() const {
for (const auto& [name, passed] : assertions) {
if (!passed) return 1; // FAIL
}
return 0; // PASS
}
```
---
### 4. SystemUtils
**Fichier**: `tests/helpers/SystemUtils.h` et `SystemUtils.cpp`
**Responsabilité**: Fonctions utilitaires système (Linux).
```cpp
// SystemUtils.h
#pragma once
#include <cstddef>
size_t getCurrentMemoryUsage();
int getOpenFileDescriptors();
float getCurrentCPUUsage();
```
```cpp
// SystemUtils.cpp
#include "SystemUtils.h"
#include <fstream>
#include <string>
#include <dirent.h>
#include <sstream>
size_t getCurrentMemoryUsage() {
// Linux: /proc/self/status -> VmRSS
std::ifstream file("/proc/self/status");
std::string line;
while (std::getline(file, line)) {
if (line.substr(0, 6) == "VmRSS:") {
std::istringstream iss(line.substr(7));
size_t kb;
iss >> kb;
return kb * 1024; // Convert to bytes
}
}
return 0;
}
int getOpenFileDescriptors() {
// Linux: /proc/self/fd
int count = 0;
DIR* dir = opendir("/proc/self/fd");
if (dir) {
struct dirent* entry;
while ((entry = readdir(dir)) != nullptr) {
count++;
}
closedir(dir);
}
return count - 2; // Exclude . and ..
}
float getCurrentCPUUsage() {
// Simplifié - retourne 0 pour l'instant
// Implémentation complète nécessite tracking du /proc/self/stat
// entre deux lectures (utime + stime delta)
return 0.0f;
}
```
---
### 5. ResourceMonitor
**Fichier**: `tests/helpers/ResourceMonitor.h` et `ResourceMonitor.cpp`
**Responsabilité**: Monitoring CPU, FD pour stress tests.
```cpp
// ResourceMonitor.h
#pragma once
#include <vector>
class ResourceMonitor {
public:
void recordFDCount(int count);
void recordCPUUsage(float percent);
int getFDAvg() const;
int getFDMax() const;
float getCPUAvg() const;
float getCPUStdDev() const;
private:
std::vector<int> fdCounts;
std::vector<float> cpuUsages;
};
```
```cpp
// ResourceMonitor.cpp
#include "ResourceMonitor.h"
#include <algorithm>
#include <numeric>
#include <cmath>
void ResourceMonitor::recordFDCount(int count) {
fdCounts.push_back(count);
}
void ResourceMonitor::recordCPUUsage(float percent) {
cpuUsages.push_back(percent);
}
int ResourceMonitor::getFDAvg() const {
if (fdCounts.empty()) return 0;
return std::accumulate(fdCounts.begin(), fdCounts.end(), 0) / fdCounts.size();
}
int ResourceMonitor::getFDMax() const {
if (fdCounts.empty()) return 0;
return *std::max_element(fdCounts.begin(), fdCounts.end());
}
float ResourceMonitor::getCPUAvg() const {
if (cpuUsages.empty()) return 0.0f;
return std::accumulate(cpuUsages.begin(), cpuUsages.end(), 0.0f) / cpuUsages.size();
}
float ResourceMonitor::getCPUStdDev() const {
if (cpuUsages.empty()) return 0.0f;
float avg = getCPUAvg();
float variance = 0.0f;
for (float cpu : cpuUsages) {
variance += std::pow(cpu - avg, 2);
}
return std::sqrt(variance / cpuUsages.size());
}
```
---
### 6. AutoCompiler
**Fichier**: `tests/helpers/AutoCompiler.h` et `AutoCompiler.cpp`
Voir détails dans `scenario_04_race_condition.md`.
---
## 🔨 CMakeLists.txt pour Tests
```cmake
# tests/CMakeLists.txt
# Helpers library (partagée par tous les tests)
add_library(test_helpers STATIC
helpers/TestMetrics.cpp
helpers/TestReporter.cpp
helpers/SystemUtils.cpp
helpers/ResourceMonitor.cpp
helpers/AutoCompiler.cpp
)
target_include_directories(test_helpers PUBLIC
${CMAKE_CURRENT_SOURCE_DIR}
)
target_link_libraries(test_helpers PUBLIC
grove_core
spdlog::spdlog
)
# Tests d'intégration
add_executable(test_01_production_hotreload integration/test_01_production_hotreload.cpp)
target_link_libraries(test_01_production_hotreload PRIVATE test_helpers grove_core)
add_executable(test_02_chaos_monkey integration/test_02_chaos_monkey.cpp)
target_link_libraries(test_02_chaos_monkey PRIVATE test_helpers grove_core)
add_executable(test_03_stress_test integration/test_03_stress_test.cpp)
target_link_libraries(test_03_stress_test PRIVATE test_helpers grove_core)
add_executable(test_04_race_condition integration/test_04_race_condition.cpp)
target_link_libraries(test_04_race_condition PRIVATE test_helpers grove_core)
add_executable(test_05_multimodule integration/test_05_multimodule.cpp)
target_link_libraries(test_05_multimodule PRIVATE test_helpers grove_core)
# Modules de test
add_library(TankModule SHARED modules/TankModule.cpp)
target_link_libraries(TankModule PRIVATE grove_core)
add_library(ProductionModule SHARED modules/ProductionModule.cpp)
target_link_libraries(ProductionModule PRIVATE grove_core)
add_library(MapModule SHARED modules/MapModule.cpp)
target_link_libraries(MapModule PRIVATE grove_core)
add_library(ChaosModule SHARED modules/ChaosModule.cpp)
target_link_libraries(ChaosModule PRIVATE grove_core)
# CTest integration
enable_testing()
add_test(NAME ProductionHotReload COMMAND test_01_production_hotreload)
add_test(NAME ChaosMonkey COMMAND test_02_chaos_monkey)
add_test(NAME StressTest COMMAND test_03_stress_test)
add_test(NAME RaceCondition COMMAND test_04_race_condition)
add_test(NAME MultiModule COMMAND test_05_multimodule)
```
---
## 🎯 Utilisation
### Compiler tous les tests
```bash
cd build
cmake -DBUILD_INTEGRATION_TESTS=ON ..
cmake --build . --target test_helpers
cmake --build . --target TankModule
cmake --build . --target ProductionModule
cmake --build .
```
### Exécuter tous les tests
```bash
ctest --output-on-failure
```
### Exécuter un test individuel
```bash
./test_01_production_hotreload
```
### Vérifier exit code
```bash
./test_01_production_hotreload
echo $? # 0 = PASS, 1 = FAIL
```
---
**Prochaine étape**: `seuils_success.md` (tous les seuils pass/fail)

309
planTI/plan_global.md Normal file
View File

@ -0,0 +1,309 @@
# Plan Global - Tests d'Intégration GroveEngine
## 🎯 Objectif
Implémenter une suite complète de tests d'intégration end-to-end pour valider la robustesse du système de hot-reload et de gestion de modules du GroveEngine.
**Contraintes:**
- ✅ 100% automatique (zéro interaction utilisateur)
- ✅ Reproductible (seed fixe pour aléatoire)
- ✅ Métriques automatiques (temps, mémoire, CPU)
- ✅ Pass/Fail clair (exit code 0/1)
- ✅ CI/CD ready
---
## 📋 Scénarios de Test (Priorité)
### Phase 1: MUST HAVE (~2-3 jours)
| Scénario | Description | Durée | Criticité |
|----------|-------------|-------|-----------|
| **1. Production Hot-Reload** | Hot-reload avec state complexe en conditions réelles | ~30s | ⭐⭐⭐ |
| **2. Chaos Monkey** | Failures aléatoires (crashes, corruptions) | ~5min | ⭐⭐⭐ |
| **3. Stress Test** | Long-running avec reloads répétés | ~10min | ⭐⭐⭐ |
### Phase 2: SHOULD HAVE (~1-2 jours)
| Scénario | Description | Durée | Criticité |
|----------|-------------|-------|-----------|
| **4. Race Condition Hunter** | Compilation concurrente + reload | ~10min | ⭐⭐ |
| **5. Multi-Module Orchestration** | Interactions entre modules | ~1min | ⭐⭐ |
### Phase 3: NICE TO HAVE (~1 jour)
| Scénario | Description | Durée | Criticité |
|----------|-------------|-------|-----------|
| **6. Error Recovery** | Crash detection + auto-recovery | ~2min | ⭐ |
| **7. Limite Tests** | Large state, long init, timeouts | ~3min | ⭐ |
| **8. Config Hot-Reload** | Changement config à la volée | ~1min | ⭐ |
---
## 🏗️ Architecture des Tests
### Structure de Fichiers
```
tests/
├─ integration/
│ ├─ test_01_production_hotreload.cpp
│ ├─ test_02_chaos_monkey.cpp
│ ├─ test_03_stress_test.cpp
│ ├─ test_04_race_condition.cpp
│ ├─ test_05_multimodule.cpp
│ ├─ test_06_error_recovery.cpp
│ ├─ test_07_limits.cpp
│ └─ test_08_config_hotreload.cpp
│
├─ modules/
│ ├─ TankModule.cpp # Module réaliste avec state complexe
│ ├─ ProductionModule.cpp # Auto-spawn entities
│ ├─ MapModule.cpp # Gestion carte/terrain
│ ├─ CrashModule.cpp # Crash contrôlé pour tests
│ └─ HeavyStateModule.cpp # State 100MB
│
└─ helpers/
├─ TestMetrics.h/.cpp # Collecte métriques (memory, CPU, FPS)
├─ TestAssertions.h # Macros ASSERT_*
├─ AutoCompiler.h/.cpp # Trigger compilation automatique
└─ TestReporter.h/.cpp # Génération rapports pass/fail
```
### Composants Communs
#### 1. TestMetrics
```cpp
class TestMetrics {
public:
void recordReloadTime(float ms);
void recordMemoryUsage(size_t bytes);
void recordFPS(float fps);
void recordCrash(const std::string& reason);
// Rapport final
void printReport();
bool meetsThresholds(const Thresholds& t);
};
```
#### 2. TestAssertions
```cpp
#define ASSERT_TRUE(cond, msg)
#define ASSERT_EQ(actual, expected)
#define ASSERT_WITHIN(actual, expected, tolerance)
#define ASSERT_LT(value, max)
#define ASSERT_GT(value, min)
```
#### 3. AutoCompiler
```cpp
class AutoCompiler {
public:
void compileModuleAsync(const std::string& moduleName);
void compileModuleSync(const std::string& moduleName);
bool isCompiling() const;
void waitForCompletion();
};
```
#### 4. TestReporter
```cpp
class TestReporter {
public:
void setScenarioName(const std::string& name);
void addMetric(const std::string& key, float value);
void addAssertion(const std::string& name, bool passed);
void printFinalReport();
int getExitCode(); // 0 = pass, 1 = fail
};
```
---
## 📊 Métriques Collectées
### Toutes les Tests
- ✅ **Temps de reload** (avg, min, max, p99)
- ✅ **Memory usage** (initial, final, growth, peak)
- ✅ **FPS** (min, avg, max, stddev)
- ✅ **Nombre de crashes** (expected vs unexpected)
- ✅ **File descriptors** (détection leaks)
### Spécifiques
- **Chaos Monkey**: Taux de recovery (%)
- **Stress Test**: Durée totale exécution
- **Race Condition**: Taux de succès compilation (%)
- **Multi-Module**: Temps isolation (impact reload d'un module sur autres)
---
## 🎯 Seuils de Succès (Thresholds)
### Production Hot-Reload
```yaml
reload_time_avg: < 500ms
reload_time_max: < 1000ms
memory_growth: < 5MB
fps_min: > 30
state_preservation: 100%
```
### Chaos Monkey
```yaml
engine_alive: true
memory_growth: < 10MB
recovery_rate: > 95%
no_deadlocks: true
```
### Stress Test
```yaml
duration: 10 minutes
reload_count: ~120 (toutes les 5s)
memory_growth: < 20MB
reload_time_p99: < 1000ms
fd_leaks: 0
```
### Race Condition
```yaml
compilation_cycles: 1000
crash_count: 0
corrupted_loads: 0
reload_success_rate: > 99%
```
### Multi-Module
```yaml
isolated_reload: true (autres modules non affectés)
execution_order_preserved: true
state_sync: 100%
```
---
## 🚀 Plan d'Exécution
### Semaine 1: Fondations + Phase 1
**Jour 1-2:**
- Créer architecture helpers (TestMetrics, Assertions, etc.)
- Implémenter TankModule (module réaliste)
- Scénario 1: Production Hot-Reload
**Jour 3:**
- Scénario 2: Chaos Monkey
**Jour 4:**
- Scénario 3: Stress Test
**Jour 5:**
- Tests + corrections Phase 1
### Semaine 2: Phase 2 + Phase 3
**Jour 6-7:**
- Scénario 4: Race Condition Hunter
- Scénario 5: Multi-Module Orchestration
**Jour 8-9:**
- Scénarios 6, 7, 8 (error recovery, limites, config)
**Jour 10:**
- Documentation finale
- CI/CD integration (GitHub Actions)
---
## 📝 Format des Rapports
### Exemple de sortie attendue
```
================================================================================
TEST: Production Hot-Reload
================================================================================
Setup:
- Module: TankModule
- Entities: 50 tanks
- Duration: 30 seconds
- Reload trigger: After 15s
Metrics:
✓ Reload time: 487ms (threshold: < 1000ms)
✓ Memory growth: 2.3MB (threshold: < 5MB)
✓ FPS min: 58 (threshold: > 30)
✓ FPS avg: 60
✓ State preservation: 50/50 tanks (100%)
Assertions:
✓ All tanks present after reload
✓ Positions preserved (error < 0.01)
✓ Velocities preserved
✓ No crashes
Result: ✅ PASSED
================================================================================
```
---
## 🔧 Commandes d'Exécution
### Compilation
```bash
cd build
cmake -DBUILD_INTEGRATION_TESTS=ON ..
cmake --build . --target integration_tests
```
### Exécution
```bash
# Tous les tests
ctest --output-on-failure
# Test individuel
./test_01_production_hotreload
./test_02_chaos_monkey --duration 300 # 5 minutes
# Avec verbosité
./test_03_stress_test --verbose
# CI mode (pas de couleurs)
./test_04_race_condition --ci
```
### Analyse
```bash
# Génération rapport JSON
./test_01_production_hotreload --output report.json
# Agrégation tous les rapports
python3 scripts/aggregate_test_reports.py
```
---
## 📌 Notes Importantes
1. **Seeds fixes**: Tous les tests avec aléatoire utilisent un seed fixe (reproductibilité)
2. **Timeouts**: Chaque test a un timeout max (évite tests infinis)
3. **Cleanup**: Tous les tests nettoient leurs ressources (fichiers temporaires, etc.)
4. **Isolation**: Chaque test peut tourner indépendamment
5. **Logs**: Niveau de log configurable (DEBUG pour dev, ERROR pour CI)
---
## 📚 Détails par Scénario
Pour les détails complets de chaque scénario, voir:
- `scenario_01_production_hotreload.md`
- `scenario_02_chaos_monkey.md`
- `scenario_03_stress_test.md`
- `scenario_04_race_condition.md`
- `scenario_05_multimodule.md`
- `scenario_06_error_recovery.md` (Phase 3)
- `scenario_07_limits.md` (Phase 3)
- `scenario_08_config_hotreload.md` (Phase 3)
---
**Dernière mise à jour**: 2025-11-13
**Statut**: Planning phase

417
planTI/scenario_01_production_hotreload.md Normal file
View File

@ -0,0 +1,417 @@
# Scénario 1: Production Hot-Reload
**Priorité**: ⭐⭐⭐ CRITIQUE
**Phase**: 1 (MUST HAVE)
**Durée estimée**: ~30 secondes
**Effort implémentation**: ~4-6 heures
---
## 🎯 Objectif
Valider que le système de hot-reload fonctionne en conditions réelles de production avec:
- State complexe (positions, vitesses, cooldowns, ordres)
- Multiples entités actives (50 tanks)
- Reload pendant l'exécution (mid-frame)
- Préservation exacte de l'état
---
## 📋 Description
### Setup Initial
1. Charger `TankModule` avec configuration initiale
2. Spawner 50 tanks avec:
- Position aléatoire dans une grille 100x100
- Vélocité aléatoire (vx, vy) entre -5 et +5
- Cooldown de tir aléatoire entre 0 et 5 secondes
- Ordre de mouvement vers une destination aléatoire
3. Exécuter pendant 15 secondes (900 frames à 60 FPS)
### Trigger Hot-Reload
1. À la frame 900 (t=15s):
- Extraire state du module via `getState()`
- Décharger le module (dlclose)
- Recompiler le module avec version modifiée (v2.0)
- Charger nouveau module (dlopen)
- Restaurer state via `setState()`
2. Mesurer temps total de reload
### Vérification Post-Reload
1. Continuer exécution pendant 15 secondes supplémentaires
2. Vérifier à chaque frame:
- Nombre de tanks = 50
- Positions dans les limites attendues
- Vélocités préservées
- Cooldowns continuent de décrémenter
- Ordres de mouvement toujours actifs
---
## 🏗️ Implémentation
### TankModule Structure
```cpp
// TankModule.h
class TankModule : public IModule {
public:
struct Tank {
float x, y; // Position
float vx, vy; // Vélocité
float cooldown; // Temps avant prochain tir
float targetX, targetY; // Destination
int id; // Identifiant unique
};
void initialize(std::shared_ptr<IDataNode> config) override;
void process(float deltaTime) override;
std::shared_ptr<IDataNode> getState() const override;
void setState(std::shared_ptr<IDataNode> state) override;
bool isIdle() const override { return true; }
private:
std::vector<Tank> tanks;
int frameCount = 0;
std::string version = "v1.0";
void updateTank(Tank& tank, float dt);
void spawnTanks(int count);
};
```
### State Format (JSON)
```json
{
"version": "v1.0",
"frameCount": 900,
"tanks": [
{
"id": 0,
"x": 23.45,
"y": 67.89,
"vx": 2.3,
"vy": -1.7,
"cooldown": 2.4,
"targetX": 80.0,
"targetY": 50.0
},
// ... 49 autres tanks
]
}
```
### Test Principal
```cpp
// test_01_production_hotreload.cpp
#include "helpers/TestMetrics.h"
#include "helpers/TestAssertions.h"
#include "helpers/TestReporter.h"
int main() {
TestReporter reporter("Production Hot-Reload");
TestMetrics metrics;
// === SETUP ===
DebugEngine engine;
engine.loadModule("TankModule", "build/modules/libTankModule.so");
// Config initiale
auto config = createJsonConfig({
{"version", "v1.0"},
{"tankCount", 50},
{"mapSize", 100}
});
engine.initializeModule("TankModule", config);
// === PHASE 1: Pre-Reload (15s) ===
std::cout << "Phase 1: Running 15s before reload...\n";
for (int i = 0; i < 900; i++) { // 15s * 60 FPS
auto start = std::chrono::high_resolution_clock::now();
engine.update(1.0f/60.0f);
auto end = std::chrono::high_resolution_clock::now();
float frameTime = std::chrono::duration<float, std::milli>(end - start).count();
metrics.recordFPS(1000.0f / frameTime);
metrics.recordMemoryUsage(getCurrentMemoryUsage());
}
// Snapshot state AVANT reload
auto preReloadState = engine.getModuleState("TankModule");
auto* jsonNode = dynamic_cast<JsonDataNode*>(preReloadState.get());
const auto& stateJson = jsonNode->getJsonData();
int tankCountBefore = stateJson["tanks"].size();
ASSERT_EQ(tankCountBefore, 50, "Should have 50 tanks before reload");
// Sauvegarder positions pour comparaison
std::vector<std::pair<float, float>> positionsBefore;
for (const auto& tank : stateJson["tanks"]) {
positionsBefore.push_back({tank["x"], tank["y"]});
}
// === HOT-RELOAD ===
std::cout << "Triggering hot-reload...\n";
auto reloadStart = std::chrono::high_resolution_clock::now();
// Modifier la version dans le code source (simulé)
modifySourceFile("tests/modules/TankModule.cpp", "v1.0", "v2.0 HOT-RELOADED");
// Recompiler
int result = system("cmake --build build --target TankModule 2>&1 | grep -v '^\\['");
ASSERT_EQ(result, 0, "Compilation should succeed");
// Le FileWatcher va détecter et recharger automatiquement
// On attend que le reload soit fait
std::this_thread::sleep_for(std::chrono::milliseconds(100));
// Ou déclencher manuellement
engine.reloadModule("TankModule");
auto reloadEnd = std::chrono::high_resolution_clock::now();
float reloadTime = std::chrono::duration<float, std::milli>(reloadEnd - reloadStart).count();
metrics.recordReloadTime(reloadTime);
reporter.addMetric("reload_time_ms", reloadTime);
// === PHASE 2: Post-Reload (15s) ===
std::cout << "Phase 2: Running 15s after reload...\n";
// Vérifier state immédiatement après reload
auto postReloadState = engine.getModuleState("TankModule");
auto* jsonNodePost = dynamic_cast<JsonDataNode*>(postReloadState.get());
const auto& stateJsonPost = jsonNodePost->getJsonData();
// Vérification 1: Nombre de tanks
int tankCountAfter = stateJsonPost["tanks"].size();
ASSERT_EQ(tankCountAfter, 50, "Should still have 50 tanks after reload");
reporter.addAssertion("tank_count_preserved", tankCountAfter == 50);
// Vérification 2: Version mise à jour
std::string versionAfter = stateJsonPost["version"];
ASSERT_TRUE(versionAfter.find("v2.0") != std::string::npos, "Version should be updated");
reporter.addAssertion("version_updated", versionAfter.find("v2.0") != std::string::npos);
// Vérification 3: Positions préservées (tolérance 0.01)
for (size_t i = 0; i < 50; i++) {
float xBefore = positionsBefore[i].first;
float yBefore = positionsBefore[i].second;
float xAfter = stateJsonPost["tanks"][i]["x"];
float yAfter = stateJsonPost["tanks"][i]["y"];
// Tolérance: pendant le reload, ~500ms se sont écoulées
// Les tanks ont bougé de velocity * 0.5s
float maxMovement = 5.0f * 0.5f; // velocity max * temps max
float distance = std::sqrt(std::pow(xAfter - xBefore, 2) + std::pow(yAfter - yBefore, 2));
ASSERT_LT(distance, maxMovement + 0.1f, "Tank position should be preserved within movement tolerance");
}
reporter.addAssertion("positions_preserved", true);
// Continuer exécution
for (int i = 0; i < 900; i++) { // 15s * 60 FPS
auto start = std::chrono::high_resolution_clock::now();
engine.update(1.0f/60.0f);
auto end = std::chrono::high_resolution_clock::now();
float frameTime = std::chrono::duration<float, std::milli>(end - start).count();
metrics.recordFPS(1000.0f / frameTime);
metrics.recordMemoryUsage(getCurrentMemoryUsage());
}
// === VÉRIFICATIONS FINALES ===
// Memory growth
size_t memGrowth = metrics.getMemoryGrowth();
ASSERT_LT(memGrowth, 5 * 1024 * 1024, "Memory growth should be < 5MB");
reporter.addMetric("memory_growth_mb", memGrowth / (1024.0f * 1024.0f));
// FPS
float minFPS = metrics.getFPSMin();
ASSERT_GT(minFPS, 30.0f, "Min FPS should be > 30");
reporter.addMetric("fps_min", minFPS);
reporter.addMetric("fps_avg", metrics.getFPSAvg());
// No crashes
reporter.addAssertion("no_crashes", true);
// === RAPPORT FINAL ===
metrics.printReport();
reporter.printFinalReport();
return reporter.getExitCode();
}
```
---
## 📊 Métriques Collectées
| Métrique | Description | Seuil |
|----------|-------------|-------|
| **reload_time_ms** | Temps total du hot-reload | < 1000ms |
| **memory_growth_mb** | Croissance mémoire (final - initial) | < 5MB |
| **fps_min** | FPS minimum observé | > 30 |
| **fps_avg** | FPS moyen sur 30s | ~60 |
| **tank_count_preserved** | Nombre de tanks identique avant/après | 50/50 |
| **positions_preserved** | Positions dans tolérance | 100% |
| **version_updated** | Version du module mise à jour | true |
---
## ✅ Critères de Succès
### MUST PASS
1. ✅ Reload time < 1000ms
2. ✅ Memory growth < 5MB
3. ✅ FPS min > 30
4. ✅ 50 tanks présents avant ET après reload
5. ✅ Positions préservées (distance < velocity * reloadTime)
6. ✅ Aucun crash
### NICE TO HAVE
1. ✅ Reload time < 500ms (optimal)
2. ✅ FPS min > 50 (très fluide)
3. ✅ Memory growth < 1MB (quasi stable)
---
## 🔧 Helpers Nécessaires
### TestMetrics.h
```cpp
class TestMetrics {
std::vector<float> fpsValues;
std::vector<size_t> memoryValues;
std::vector<float> reloadTimes;
size_t initialMemory;
public:
void recordFPS(float fps) { fpsValues.push_back(fps); }
void recordMemoryUsage(size_t bytes) { memoryValues.push_back(bytes); }
void recordReloadTime(float ms) { reloadTimes.push_back(ms); }
float getFPSMin() const { return *std::min_element(fpsValues.begin(), fpsValues.end()); }
float getFPSAvg() const { return std::accumulate(fpsValues.begin(), fpsValues.end(), 0.0f) / fpsValues.size(); }
size_t getMemoryGrowth() const { return memoryValues.back() - initialMemory; }
void printReport() const;
};
```
### Utility Functions
```cpp
size_t getCurrentMemoryUsage() {
// Linux: /proc/self/status
std::ifstream file("/proc/self/status");
std::string line;
while (std::getline(file, line)) {
if (line.substr(0, 6) == "VmRSS:") {
size_t kb = std::stoi(line.substr(7));
return kb * 1024;
}
}
return 0;
}
void modifySourceFile(const std::string& path, const std::string& oldStr, const std::string& newStr) {
std::ifstream input(path);
std::string content((std::istreambuf_iterator<char>(input)), std::istreambuf_iterator<char>());
input.close();
size_t pos = content.find(oldStr);
if (pos != std::string::npos) {
content.replace(pos, oldStr.length(), newStr);
}
std::ofstream output(path);
output << content;
}
```
---
## 🐛 Cas d'Erreur Attendus
| Erreur | Cause | Action |
|--------|-------|--------|
| Tank count mismatch | État corrompu | FAIL - état mal sauvegardé/restauré |
| Position out of bounds | Calcul incorrect | FAIL - bug dans updateTank() |
| Reload time > 1s | Compilation lente | FAIL - optimiser build |
| Memory growth > 5MB | Memory leak | FAIL - vérifier destructeurs |
| FPS < 30 | Reload bloque trop | FAIL - optimiser waitForCleanState |
---
## 📝 Output Attendu
```
================================================================================
TEST: Production Hot-Reload
================================================================================
Phase 1: Running 15s before reload...
[900 frames processed]
State snapshot:
- Tanks: 50
- Version: v1.0
- Frame: 900
Triggering hot-reload...
[Compilation OK]
[Reload completed in 487ms]
Verification:
✓ Tank count: 50/50
✓ Version updated: v2.0 HOT-RELOADED
✓ Positions preserved: 50/50 (max error: 0.003)
Phase 2: Running 15s after reload...
[900 frames processed]
================================================================================
METRICS
================================================================================
Reload time: 487ms (threshold: < 1000ms) ✓
Memory growth: 2.3MB (threshold: < 5MB) ✓
FPS min: 58 (threshold: > 30) ✓
FPS avg: 60
FPS max: 62
================================================================================
ASSERTIONS
================================================================================
✓ tank_count_preserved
✓ version_updated
✓ positions_preserved
✓ no_crashes
Result: ✅ PASSED
================================================================================
```
---
## 📅 Planning
**Jour 1 (4h):**
- Implémenter TankModule avec state complexe
- Implémenter helpers (TestMetrics, assertions)
**Jour 2 (2h):**
- Implémenter test_01_production_hotreload.cpp
- Debug + validation
---
**Prochaine étape**: `scenario_02_chaos_monkey.md`

468
planTI/scenario_02_chaos_monkey.md Normal file
View File

@ -0,0 +1,468 @@
# Scénario 2: Chaos Monkey
**Priorité**: ⭐⭐⭐ CRITIQUE
**Phase**: 1 (MUST HAVE)
**Durée estimée**: ~5 minutes
**Effort implémentation**: ~6-8 heures
---
## 🎯 Objectif
Valider la robustesse du système face à des failures aléatoires et sa capacité à:
- Détecter les crashes
- Récupérer automatiquement
- Maintenir la stabilité mémoire
- Éviter les deadlocks
- Logger les erreurs correctement
**Inspiré de**: Netflix Chaos Monkey (tester en cassant aléatoirement)
---
## 📋 Description
### Principe
Exécuter l'engine pendant 5 minutes avec un module qui génère des events aléatoires de failure:
- 30% chance de hot-reload par seconde
- 10% chance de crash dans `process()`
- 10% chance de state corrompu
- 5% chance de config invalide
- 45% chance de fonctionnement normal
### Comportement Attendu
L'engine doit:
1. Détecter chaque crash
2. Logger l'erreur avec stack trace
3. Récupérer en rechargeant le module
4. Continuer l'exécution
5. Ne jamais deadlock
6. Memory usage stable (< 10MB growth)
---
## 🏗️ Implémentation
### ChaosModule Structure
```cpp
// ChaosModule.h
class ChaosModule : public IModule {
public:
void initialize(std::shared_ptr<IDataNode> config) override;
void process(float deltaTime) override;
std::shared_ptr<IDataNode> getState() const override;
void setState(std::shared_ptr<IDataNode> state) override;
bool isIdle() const override { return !isProcessing; }
private:
std::mt19937 rng;
int frameCount = 0;
int crashCount = 0;
int corruptionCount = 0;
bool isProcessing = false;
// Configuration du chaos
float hotReloadProbability = 0.30f;
float crashProbability = 0.10f;
float corruptionProbability = 0.10f;
float invalidConfigProbability = 0.05f;
// Simulations de failures
void maybeHotReload();
void maybeCrash();
void maybeCorruptState();
void maybeInvalidConfig();
};
```
### Process Logic
```cpp
void ChaosModule::process(float deltaTime) {
isProcessing = true;
frameCount++;
// Générer event aléatoire (1 fois par seconde = 60 frames)
if (frameCount % 60 == 0) {
float roll = (rng() % 100) / 100.0f;
if (roll < hotReloadProbability) {
// Signal pour hot-reload (via flag dans state)
logger->info("🎲 Chaos: Triggering HOT-RELOAD");
// Note: Le test externe déclenchera le reload
}
else if (roll < hotReloadProbability + crashProbability) {
// CRASH INTENTIONNEL
logger->warn("🎲 Chaos: Triggering CRASH");
crashCount++;
throw std::runtime_error("Intentional crash for chaos testing");
}
else if (roll < hotReloadProbability + crashProbability + corruptionProbability) {
// CORRUPTION DE STATE
logger->warn("🎲 Chaos: Corrupting STATE");
corruptionCount++;
// Modifier state de façon invalide (sera détecté à getState)
}
else if (roll < hotReloadProbability + crashProbability + corruptionProbability + invalidConfigProbability) {
// CONFIG INVALIDE (simulé)
logger->warn("🎲 Chaos: Invalid CONFIG request");
// Le test externe tentera setState avec config invalide
}
// Sinon: fonctionnement normal (45%)
}
isProcessing = false;
}
```
### State Format
```json
{
"frameCount": 18000,
"crashCount": 12,
"corruptionCount": 8,
"hotReloadCount": 90,
"seed": 42,
"isCorrupted": false
}
```
### Test Principal
```cpp
// test_02_chaos_monkey.cpp
#include "helpers/TestMetrics.h"
#include "helpers/TestReporter.h"
#include <csignal>
// Global pour catch segfault
static bool engineCrashed = false;
void signalHandler(int signal) {
if (signal == SIGSEGV || signal == SIGABRT) {
engineCrashed = true;
}
}
int main() {
TestReporter reporter("Chaos Monkey");
TestMetrics metrics;
// Setup signal handlers
std::signal(SIGSEGV, signalHandler);
std::signal(SIGABRT, signalHandler);
// === SETUP ===
DebugEngine engine;
engine.loadModule("ChaosModule", "build/modules/libChaosModule.so");
auto config = createJsonConfig({
{"seed", 42}, // Reproductible
{"hotReloadProbability", 0.30},
{"crashProbability", 0.10},
{"corruptionProbability", 0.10},
{"invalidConfigProbability", 0.05}
});
engine.initializeModule("ChaosModule", config);
// === CHAOS LOOP (5 minutes = 18000 frames) ===
std::cout << "Starting Chaos Monkey (5 minutes)...\n";
int totalFrames = 18000; // 5 * 60 * 60
int crashesDetected = 0;
int reloadsTriggered = 0;
int recoverySuccesses = 0;
bool hadDeadlock = false;
auto testStart = std::chrono::high_resolution_clock::now();
for (int frame = 0; frame < totalFrames; frame++) {
auto frameStart = std::chrono::high_resolution_clock::now();
try {
// Update engine
engine.update(1.0f / 60.0f);
// Check si le module demande un hot-reload
auto state = engine.getModuleState("ChaosModule");
auto* jsonNode = dynamic_cast<JsonDataNode*>(state.get());
const auto& stateJson = jsonNode->getJsonData();
// Simuler hot-reload si demandé (aléatoire 30%)
if (frame % 60 == 0) { // Check toutes les secondes
float roll = (rand() % 100) / 100.0f;
if (roll < 0.30f) {
std::cout << " [Frame " << frame << "] Hot-reload triggered\n";
auto reloadStart = std::chrono::high_resolution_clock::now();
engine.reloadModule("ChaosModule");
reloadsTriggered++;
auto reloadEnd = std::chrono::high_resolution_clock::now();
float reloadTime = std::chrono::duration<float, std::milli>(reloadEnd - reloadStart).count();
metrics.recordReloadTime(reloadTime);
}
}
} catch (const std::exception& e) {
// CRASH DÉTECTÉ
crashesDetected++;
std::cout << " [Frame " << frame << "] ⚠️ Crash detected: " << e.what() << "\n";
// Tentative de recovery
try {
std::cout << " [Frame " << frame << "] 🔄 Attempting recovery...\n";
// Recharger le module
engine.reloadModule("ChaosModule");
// Réinitialiser avec state par défaut
engine.initializeModule("ChaosModule", config);
recoverySuccesses++;
std::cout << " [Frame " << frame << "] ✅ Recovery successful\n";
} catch (const std::exception& recoveryError) {
std::cout << " [Frame " << frame << "] ❌ Recovery FAILED: " << recoveryError.what() << "\n";
reporter.addAssertion("recovery_failed", false);
break; // Arrêter le test
}
}
// Métriques
auto frameEnd = std::chrono::high_resolution_clock::now();
float frameTime = std::chrono::duration<float, std::milli>(frameEnd - frameStart).count();
metrics.recordFPS(1000.0f / frameTime);
metrics.recordMemoryUsage(getCurrentMemoryUsage());
// Deadlock detection (frame > 100ms)
if (frameTime > 100.0f) {
std::cout << " [Frame " << frame << "] ⚠️ Potential deadlock (frame time: " << frameTime << "ms)\n";
hadDeadlock = true;
}
// Progress (toutes les 3000 frames = 50s)
if (frame % 3000 == 0 && frame > 0) {
float elapsedMin = frame / 3600.0f;
std::cout << "Progress: " << elapsedMin << "/5.0 minutes (" << (frame * 100 / totalFrames) << "%)\n";
}
}
auto testEnd = std::chrono::high_resolution_clock::now();
float totalDuration = std::chrono::duration<float>(testEnd - testStart).count();
// === VÉRIFICATIONS FINALES ===
// Engine toujours vivant
bool engineAlive = !engineCrashed;
ASSERT_TRUE(engineAlive, "Engine should still be alive");
reporter.addAssertion("engine_alive", engineAlive);
// Pas de deadlocks
ASSERT_FALSE(hadDeadlock, "Should not have deadlocks");
reporter.addAssertion("no_deadlocks", !hadDeadlock);
// Recovery rate > 95%
float recoveryRate = (crashesDetected > 0) ? (recoverySuccesses * 100.0f / crashesDetected) : 100.0f;
ASSERT_GT(recoveryRate, 95.0f, "Recovery rate should be > 95%");
reporter.addMetric("recovery_rate_percent", recoveryRate);
// Memory growth < 10MB
size_t memGrowth = metrics.getMemoryGrowth();
ASSERT_LT(memGrowth, 10 * 1024 * 1024, "Memory growth should be < 10MB");
reporter.addMetric("memory_growth_mb", memGrowth / (1024.0f * 1024.0f));
// Durée totale proche de 5 minutes (tolérance ±10s)
ASSERT_WITHIN(totalDuration, 300.0f, 10.0f, "Total duration should be ~5 minutes");
reporter.addMetric("total_duration_sec", totalDuration);
// Statistiques
reporter.addMetric("crashes_detected", crashesDetected);
reporter.addMetric("reloads_triggered", reloadsTriggered);
reporter.addMetric("recovery_successes", recoverySuccesses);
std::cout << "\n";
std::cout << "================================================================================\n";
std::cout << "CHAOS MONKEY STATISTICS\n";
std::cout << "================================================================================\n";
std::cout << " Total frames: " << totalFrames << "\n";
std::cout << " Duration: " << totalDuration << "s\n";
std::cout << " Crashes detected: " << crashesDetected << "\n";
std::cout << " Reloads triggered: " << reloadsTriggered << "\n";
std::cout << " Recovery successes: " << recoverySuccesses << "\n";
std::cout << " Recovery rate: " << recoveryRate << "%\n";
std::cout << " Memory growth: " << (memGrowth / (1024.0f * 1024.0f)) << " MB\n";
std::cout << " Had deadlocks: " << (hadDeadlock ? "YES ❌" : "NO ✅") << "\n";
std::cout << "================================================================================\n\n";
// === RAPPORT FINAL ===
metrics.printReport();
reporter.printFinalReport();
return reporter.getExitCode();
}
```
---
## 📊 Métriques Collectées
| Métrique | Description | Seuil |
|----------|-------------|-------|
| **engine_alive** | Engine toujours vivant à la fin | true |
| **no_deadlocks** | Aucun deadlock détecté | true |
| **recovery_rate_percent** | % de crashes récupérés | > 95% |
| **memory_growth_mb** | Croissance mémoire totale | < 10MB |
| **total_duration_sec** | Durée totale du test | ~300s (±10s) |
| **crashes_detected** | Nombre de crashes détectés | N/A (info) |
| **reloads_triggered** | Nombre de hot-reloads | ~90 (30% * 300s) |
| **recovery_successes** | Nombre de recoveries réussies | ~crashes_detected |
---
## ✅ Critères de Succès
### MUST PASS
1. ✅ Engine toujours vivant après 5 minutes
2. ✅ Aucun deadlock
3. ✅ Recovery rate > 95%
4. ✅ Memory growth < 10MB
5. ✅ Logs contiennent tous les crashes (avec stack trace si possible)
### NICE TO HAVE
1. ✅ Recovery rate = 100% (aucun échec)
2. ✅ Memory growth < 5MB
3. ✅ Reload time moyen < 500ms même pendant chaos
---
## 🔧 Recovery Strategy
### Dans DebugEngine::update()
```cpp
void DebugEngine::update(float deltaTime) {
try {
moduleSystem->update(deltaTime);
} catch (const std::exception& e) {
// CRASH DÉTECTÉ
logger->error("❌ Module crashed: {}", e.what());
// Tentative de recovery automatique
try {
logger->info("🔄 Attempting automatic recovery...");
// 1. Extraire le module défaillant
auto failedModule = moduleSystem->extractModule();
// 2. Recharger depuis .so
std::string moduleName = "ChaosModule"; // À généraliser
reloadModule(moduleName);
// 3. Réinitialiser avec config par défaut
auto defaultConfig = createDefaultConfig();
auto newModule = moduleLoader->getModule(moduleName);
newModule->initialize(defaultConfig);
// 4. Ré-enregistrer
moduleSystem->registerModule(moduleName, std::move(newModule));
logger->info("✅ Recovery successful");
} catch (const std::exception& recoveryError) {
logger->critical("❌ Recovery failed: {}", recoveryError.what());
throw; // Re-throw si recovery impossible
}
}
}
```
---
## 🐛 Cas d'Erreur Attendus
| Erreur | Cause | Action |
|--------|-------|--------|
| Crash non récupéré | Recovery logic bugué | FAIL - fixer recovery |
| Deadlock | Mutex lock dans crash handler | FAIL - review locks |
| Memory leak > 10MB | Module pas correctement nettoyé | FAIL - fix destructors |
| Duration >> 300s | Reloads trop longs | WARNING - optimiser |
---
## 📝 Output Attendu
```
================================================================================
TEST: Chaos Monkey
================================================================================
Starting Chaos Monkey (5 minutes)...
[Frame 60] Hot-reload triggered
[Frame 180] ⚠️ Crash detected: Intentional crash for chaos testing
[Frame 180] 🔄 Attempting recovery...
[Frame 180] ✅ Recovery successful
[Frame 240] Hot-reload triggered
...
Progress: 1.0/5.0 minutes (20%)
[Frame 3420] ⚠️ Crash detected: Intentional crash for chaos testing
[Frame 3420] 🔄 Attempting recovery...
[Frame 3420] ✅ Recovery successful
...
Progress: 5.0/5.0 minutes (100%)
================================================================================
CHAOS MONKEY STATISTICS
================================================================================
Total frames: 18000
Duration: 302.4s
Crashes detected: 28
Reloads triggered: 89
Recovery successes: 28
Recovery rate: 100%
Memory growth: 3.2 MB
Had deadlocks: NO ✅
================================================================================
METRICS
================================================================================
Engine alive: true ✓
No deadlocks: true ✓
Recovery rate: 100% (threshold: > 95%) ✓
Memory growth: 3.2 MB (threshold: < 10MB) ✓
Total duration: 302.4s (expected: ~300s) ✓
ASSERTIONS
================================================================================
✓ engine_alive
✓ no_deadlocks
✓ recovery_rate > 95%
✓ memory_stable
Result: ✅ PASSED
================================================================================
```
---
## 📅 Planning
**Jour 1 (4h):**
- Implémenter ChaosModule avec probabilités configurables
- Implémenter recovery logic dans DebugEngine
**Jour 2 (4h):**
- Implémenter test_02_chaos_monkey.cpp
- Signal handling (SIGSEGV, SIGABRT)
- Deadlock detection
- Debug + validation
---
**Prochaine étape**: `scenario_03_stress_test.md`

507
planTI/scenario_03_stress_test.md Normal file
View File

@ -0,0 +1,507 @@
# Scénario 3: Stress Test Long-Running
**Priorité**: ⭐⭐⭐ CRITIQUE
**Phase**: 1 (MUST HAVE)
**Durée estimée**: ~10 minutes (extensible à 1h pour nightly)
**Effort implémentation**: ~4-6 heures
---
## 🎯 Objectif
Valider la stabilité du système sur une longue durée avec:
- Memory leaks détectables
- Performance degradation mesurable
- File descriptor leaks
- CPU usage stable
- Hot-reload répétés sans dégradation
**But**: Prouver que le système peut tourner en production 24/7
---
## 📋 Description
### Setup
- Charger 3 modules simultanément:
- `TankModule` (50 tanks actifs)
- `ProductionModule` (spawn 1 tank/seconde)
- `MapModule` (grille 200x200)
- Exécuter à 60 FPS constant pendant 10 minutes
- Hot-reload round-robin toutes les 5 secondes (120 reloads total)
### Métriques Critiques
1. **Memory**: Croissance < 20MB sur 10 minutes
2. **CPU**: Usage stable (variation < 10%)
3. **FPS**: Minimum > 30 (jamais de freeze)
4. **Reload latency**: P99 < 1s (même après 120 reloads)
5. **File descriptors**: Aucun leak
---
## 🏗️ Implémentation
### Modules de Test
#### TankModule (déjà existant)
```cpp
// 50 tanks qui bougent en continu
class TankModule : public IModule {
std::vector<Tank> tanks; // 50 tanks
void process(float dt) override {
for (auto& tank : tanks) {
tank.position += tank.velocity * dt;
}
}
};
```
#### ProductionModule
```cpp
class ProductionModule : public IModule {
public:
void process(float deltaTime) override {
timeSinceLastSpawn += deltaTime;
// Spawner 1 tank par seconde
if (timeSinceLastSpawn >= 1.0f) {
spawnTank();
timeSinceLastSpawn -= 1.0f;
}
}
std::shared_ptr<IDataNode> getState() const override {
auto state = std::make_shared<JsonDataNode>();
auto& json = state->getJsonData();
json["tankCount"] = tankCount;
json["timeSinceLastSpawn"] = timeSinceLastSpawn;
nlohmann::json tanksJson = nlohmann::json::array();
for (const auto& tank : spawnedTanks) {
tanksJson.push_back({
{"id", tank.id},
{"spawnTime", tank.spawnTime}
});
}
json["spawnedTanks"] = tanksJson;
return state;
}
private:
int tankCount = 0;
float timeSinceLastSpawn = 0.0f;
std::vector<SpawnedTank> spawnedTanks;
void spawnTank() {
tankCount++;
spawnedTanks.push_back({tankCount, getCurrentTime()});
logger->debug("Spawned tank #{}", tankCount);
}
};
```
#### MapModule
```cpp
class MapModule : public IModule {
public:
void initialize(std::shared_ptr<IDataNode> config) override {
int size = config->getInt("mapSize", 200);
grid.resize(size * size, 0); // Grille 200x200 = 40k cells
}
void process(float deltaTime) override {
// Update grille (simuler fog of war ou autre)
for (size_t i = 0; i < grid.size(); i += 100) {
grid[i] = (grid[i] + 1) % 256;
}
}
std::shared_ptr<IDataNode> getState() const override {
auto state = std::make_shared<JsonDataNode>();
auto& json = state->getJsonData();
json["mapSize"] = std::sqrt(grid.size());
// Ne pas sérialiser toute la grille (trop gros)
json["gridChecksum"] = computeChecksum(grid);
return state;
}
private:
std::vector<uint8_t> grid;
uint32_t computeChecksum(const std::vector<uint8_t>& data) const {
uint32_t sum = 0;
for (auto val : data) sum += val;
return sum;
}
};
```
### Test Principal
```cpp
// test_03_stress_test.cpp
#include "helpers/TestMetrics.h"
#include "helpers/TestReporter.h"
#include "helpers/ResourceMonitor.h"
int main(int argc, char* argv[]) {
// Durée configurable (10 min par défaut, 1h pour nightly)
int durationMinutes = 10;
if (argc > 1 && std::string(argv[1]) == "--nightly") {
durationMinutes = 60;
}
int totalFrames = durationMinutes * 60 * 60; // min * sec * fps
int reloadIntervalFrames = 5 * 60; // 5 secondes
TestReporter reporter("Stress Test Long-Running");
TestMetrics metrics;
ResourceMonitor resMonitor;
std::cout << "================================================================================\n";
std::cout << "STRESS TEST: " << durationMinutes << " minutes\n";
std::cout << "================================================================================\n\n";
// === SETUP ===
DebugEngine engine;
// Charger 3 modules
engine.loadModule("TankModule", "build/modules/libTankModule.so");
engine.loadModule("ProductionModule", "build/modules/libProductionModule.so");
engine.loadModule("MapModule", "build/modules/libMapModule.so");
// Configurations
auto tankConfig = createJsonConfig({{"tankCount", 50}});
auto prodConfig = createJsonConfig({{"spawnRate", 1.0}});
auto mapConfig = createJsonConfig({{"mapSize", 200}});
engine.initializeModule("TankModule", tankConfig);
engine.initializeModule("ProductionModule", prodConfig);
engine.initializeModule("MapModule", mapConfig);
// Baseline metrics
size_t baselineMemory = getCurrentMemoryUsage();
int baselineFDs = getOpenFileDescriptors();
float baselineCPU = getCurrentCPUUsage();
std::cout << "Baseline:\n";
std::cout << " Memory: " << (baselineMemory / (1024.0f * 1024.0f)) << " MB\n";
std::cout << " FDs: " << baselineFDs << "\n";
std::cout << " CPU: " << baselineCPU << "%\n\n";
// === STRESS LOOP ===
std::vector<std::string> moduleNames = {"TankModule", "ProductionModule", "MapModule"};
int currentModuleIndex = 0;
int reloadCount = 0;
auto testStart = std::chrono::high_resolution_clock::now();
for (int frame = 0; frame < totalFrames; frame++) {
auto frameStart = std::chrono::high_resolution_clock::now();
// Update engine
engine.update(1.0f / 60.0f);
// Hot-reload round-robin toutes les 5 secondes
if (frame > 0 && frame % reloadIntervalFrames == 0) {
std::string moduleName = moduleNames[currentModuleIndex];
std::cout << "[" << (frame / 3600.0f) << "min] Hot-reloading " << moduleName << "...\n";
auto reloadStart = std::chrono::high_resolution_clock::now();
engine.reloadModule(moduleName);
reloadCount++;
auto reloadEnd = std::chrono::high_resolution_clock::now();
float reloadTime = std::chrono::duration<float, std::milli>(reloadEnd - reloadStart).count();
metrics.recordReloadTime(reloadTime);
std::cout << " → Completed in " << reloadTime << "ms\n";
// Rotate module
currentModuleIndex = (currentModuleIndex + 1) % moduleNames.size();
}
// Métriques (échantillonner toutes les 60 frames = 1 seconde)
if (frame % 60 == 0) {
size_t currentMemory = getCurrentMemoryUsage();
int currentFDs = getOpenFileDescriptors();
float currentCPU = getCurrentCPUUsage();
metrics.recordMemoryUsage(currentMemory);
resMonitor.recordFDCount(currentFDs);
resMonitor.recordCPUUsage(currentCPU);
}
// FPS (chaque frame)
auto frameEnd = std::chrono::high_resolution_clock::now();
float frameTime = std::chrono::duration<float, std::milli>(frameEnd - frameStart).count();
metrics.recordFPS(1000.0f / frameTime);
// Progress (toutes les minutes)
if (frame % 3600 == 0 && frame > 0) {
int elapsedMin = frame / 3600;
std::cout << "Progress: " << elapsedMin << "/" << durationMinutes << " minutes\n";
// Stats intermédiaires
size_t currentMemory = getCurrentMemoryUsage();
float memGrowth = (currentMemory - baselineMemory) / (1024.0f * 1024.0f);
std::cout << " Memory growth: " << memGrowth << " MB\n";
std::cout << " FPS (last min): min=" << metrics.getFPSMinLast60s()
<< " avg=" << metrics.getFPSAvgLast60s() << "\n";
std::cout << " Reload avg: " << metrics.getReloadTimeAvg() << "ms\n\n";
}
}
auto testEnd = std::chrono::high_resolution_clock::now();
float totalDuration = std::chrono::duration<float>(testEnd - testStart).count();
// === VÉRIFICATIONS FINALES ===
size_t finalMemory = getCurrentMemoryUsage();
size_t memGrowth = finalMemory - baselineMemory;
int finalFDs = getOpenFileDescriptors();
int fdLeak = finalFDs - baselineFDs;
float avgCPU = resMonitor.getCPUAvg();
float cpuStdDev = resMonitor.getCPUStdDev();
// Assertions
ASSERT_LT(memGrowth, 20 * 1024 * 1024, "Memory growth should be < 20MB");
reporter.addMetric("memory_growth_mb", memGrowth / (1024.0f * 1024.0f));
ASSERT_EQ(fdLeak, 0, "Should have no file descriptor leaks");
reporter.addMetric("fd_leak", fdLeak);
float fpsMin = metrics.getFPSMin();
ASSERT_GT(fpsMin, 30.0f, "FPS min should be > 30");
reporter.addMetric("fps_min", fpsMin);
reporter.addMetric("fps_avg", metrics.getFPSAvg());
float reloadP99 = metrics.getReloadTimeP99();
ASSERT_LT(reloadP99, 1000.0f, "Reload P99 should be < 1000ms");
reporter.addMetric("reload_time_p99_ms", reloadP99);
ASSERT_LT(cpuStdDev, 10.0f, "CPU usage should be stable (stddev < 10%)");
reporter.addMetric("cpu_avg_percent", avgCPU);
reporter.addMetric("cpu_stddev_percent", cpuStdDev);
reporter.addMetric("total_reloads", reloadCount);
reporter.addMetric("total_duration_sec", totalDuration);
// === RAPPORT FINAL ===
std::cout << "\n";
std::cout << "================================================================================\n";
std::cout << "STRESS TEST SUMMARY\n";
std::cout << "================================================================================\n";
std::cout << " Duration: " << totalDuration << "s (" << (totalDuration / 60.0f) << " min)\n";
std::cout << " Total reloads: " << reloadCount << "\n";
std::cout << " Memory growth: " << (memGrowth / (1024.0f * 1024.0f)) << " MB\n";
std::cout << " FD leak: " << fdLeak << "\n";
std::cout << " FPS min/avg/max: " << fpsMin << " / " << metrics.getFPSAvg() << " / " << metrics.getFPSMax() << "\n";
std::cout << " Reload avg/p99: " << metrics.getReloadTimeAvg() << "ms / " << reloadP99 << "ms\n";
std::cout << " CPU avg±stddev: " << avgCPU << "% ± " << cpuStdDev << "%\n";
std::cout << "================================================================================\n\n";
metrics.printReport();
reporter.printFinalReport();
return reporter.getExitCode();
}
```
---
## 📊 Métriques Collectées
| Métrique | Description | Seuil (10 min) | Seuil (1h) |
|----------|-------------|----------------|------------|
| **memory_growth_mb** | Croissance mémoire totale | < 20MB | < 100MB |
| **fd_leak** | File descriptors ouverts en trop | 0 | 0 |
| **fps_min** | FPS minimum observé | > 30 | > 30 |
| **fps_avg** | FPS moyen | ~60 | ~60 |
| **reload_time_p99_ms** | Latence P99 des reloads | < 1000ms | < 1000ms |
| **cpu_avg_percent** | CPU moyen | N/A (info) | N/A (info) |
| **cpu_stddev_percent** | Stabilité CPU | < 10% | < 10% |
| **total_reloads** | Nombre total de reloads | ~120 | ~720 |
---
## ✅ Critères de Succès
### MUST PASS (10 minutes)
1. ✅ Memory growth < 20MB
2. ✅ FD leak = 0
3. ✅ FPS min > 30
4. ✅ Reload P99 < 1000ms
5. ✅ CPU stable (stddev < 10%)
6. ✅ Aucun crash
### MUST PASS (1 heure nightly)
1. ✅ Memory growth < 100MB
2. ✅ FD leak = 0
3. ✅ FPS min > 30
4. ✅ Reload P99 < 1000ms (pas de dégradation)
5. ✅ CPU stable (stddev < 10%)
6. ✅ Aucun crash
---
## 🔧 Helpers Nécessaires
### ResourceMonitor
```cpp
// helpers/ResourceMonitor.h
class ResourceMonitor {
public:
void recordFDCount(int count) {
fdCounts.push_back(count);
}
void recordCPUUsage(float percent) {
cpuUsages.push_back(percent);
}
float getCPUAvg() const {
return std::accumulate(cpuUsages.begin(), cpuUsages.end(), 0.0f) / cpuUsages.size();
}
float getCPUStdDev() const {
float avg = getCPUAvg();
float variance = 0.0f;
for (float cpu : cpuUsages) {
variance += std::pow(cpu - avg, 2);
}
return std::sqrt(variance / cpuUsages.size());
}
private:
std::vector<int> fdCounts;
std::vector<float> cpuUsages;
};
```
### System Utilities
```cpp
// helpers/SystemUtils.h
int getOpenFileDescriptors() {
// Linux: /proc/self/fd
int count = 0;
DIR* dir = opendir("/proc/self/fd");
if (dir) {
while (readdir(dir)) count++;
closedir(dir);
}
return count - 2; // Exclude . and ..
}
float getCurrentCPUUsage() {
// Linux: /proc/self/stat
std::ifstream stat("/proc/self/stat");
std::string line;
std::getline(stat, line);
// Parse utime + stime (fields 14 & 15)
// Comparer avec previous reading pour obtenir %
// Simplifié ici, voir impl complète
return 0.0f; // Placeholder
}
```
---
## 📝 Output Attendu (10 minutes)
```
================================================================================
STRESS TEST: 10 minutes
================================================================================
Baseline:
Memory: 45.2 MB
FDs: 12
CPU: 2.3%
[0.08min] Hot-reloading TankModule...
→ Completed in 423ms
[0.17min] Hot-reloading ProductionModule...
→ Completed in 389ms
Progress: 1/10 minutes
Memory growth: 1.2 MB
FPS (last min): min=59 avg=60
Reload avg: 405ms
Progress: 2/10 minutes
Memory growth: 2.1 MB
FPS (last min): min=58 avg=60
Reload avg: 412ms
...
Progress: 10/10 minutes
Memory growth: 8.7 MB
FPS (last min): min=59 avg=60
Reload avg: 418ms
================================================================================
STRESS TEST SUMMARY
================================================================================
Duration: 601.2s (10.0 min)
Total reloads: 120
Memory growth: 8.7 MB
FD leak: 0
FPS min/avg/max: 58 / 60 / 62
Reload avg/p99: 415ms / 687ms
CPU avg±stddev: 12.3% ± 3.2%
================================================================================
METRICS
================================================================================
Memory growth: 8.7 MB (threshold: < 20MB) ✓
FD leak: 0 (threshold: 0) ✓
FPS min: 58 (threshold: > 30) ✓
Reload P99: 687ms (threshold: < 1000ms) ✓
CPU stable: 3.2% (threshold: < 10%) ✓
Result: ✅ PASSED
================================================================================
```
---
## 🐛 Cas d'Erreur Attendus
| Erreur | Cause | Action |
|--------|-------|--------|
| Memory growth > 20MB | Memory leak dans module | FAIL - fix destructors |
| FD leak > 0 | dlopen/dlclose déséquilibré | FAIL - fix ModuleLoader |
| FPS degradation | Performance regression | FAIL - profile + optimize |
| Reload P99 croissant | Fragmentation mémoire | WARNING - investigate |
| CPU instable | Busy loop ou GC | FAIL - fix algorithm |
---
## 📅 Planning
**Jour 1 (3h):**
- Implémenter ProductionModule et MapModule
- Implémenter ResourceMonitor helper
**Jour 2 (3h):**
- Implémenter test_03_stress_test.cpp
- System utilities (FD count, CPU usage)
- Debug + validation
---
**Prochaine étape**: `scenario_04_race_condition.md`

494
planTI/scenario_04_race_condition.md Normal file
View File

@ -0,0 +1,494 @@
# Scénario 4: Race Condition Hunter
**Priorité**: ⭐⭐ IMPORTANT
**Phase**: 2 (SHOULD HAVE)
**Durée estimée**: ~10 minutes (1000 compilations)
**Effort implémentation**: ~6-8 heures
---
## 🎯 Objectif
Détecter et valider la robustesse face aux race conditions lors de la compilation concurrente:
- FileWatcher détecte changements pendant compilation
- File stability check fonctionne
- Aucun .so corrompu chargé
- Aucun deadlock entre threads
- 100% success rate des reloads
**C'est le test qui a motivé le fix de la race condition initiale !**
---
## 📋 Description
### Setup
1. Thread 1 (Compiler): Recompile `TestModule.so` toutes les 300ms
2. Thread 2 (FileWatcher): Détecte changements et déclenche reload
3. Thread 3 (Engine): Exécute `process()` en tight loop à 60 FPS
4. Durée: 1000 cycles de compilation (~5 minutes)
### Comportements à Tester
- **File stability check**: Attend que le fichier soit stable avant reload
- **Size verification**: Vérifie que le .so copié est complet
- **Concurrent access**: Pas de corruption pendant dlopen/dlclose
- **Error handling**: Détecte et récupère des .so incomplets
---
## 🏗️ Implémentation
### AutoCompiler Helper
```cpp
// helpers/AutoCompiler.h
class AutoCompiler {
public:
AutoCompiler(const std::string& moduleName, const std::string& buildDir)
: moduleName(moduleName), buildDir(buildDir), isRunning(false) {}
void start(int iterations, int intervalMs) {
isRunning = true;
compilerThread = std::thread([this, iterations, intervalMs]() {
for (int i = 0; i < iterations && isRunning; i++) {
compile(i);
std::this_thread::sleep_for(std::chrono::milliseconds(intervalMs));
}
});
}
void stop() {
isRunning = false;
if (compilerThread.joinable()) {
compilerThread.join();
}
}
int getSuccessCount() const { return successCount; }
int getFailureCount() const { return failureCount; }
int getCurrentIteration() const { return currentIteration; }
private:
std::string moduleName;
std::string buildDir;
std::atomic<bool> isRunning;
std::atomic<int> successCount{0};
std::atomic<int> failureCount{0};
std::atomic<int> currentIteration{0};
std::thread compilerThread;
void compile(int iteration) {
currentIteration = iteration;
// Modifier source pour forcer recompilation
modifySourceVersion(iteration);
// Compiler
std::string cmd = "cmake --build " + buildDir + " --target " + moduleName + " 2>&1";
int result = system(cmd.c_str());
if (result == 0) {
successCount++;
} else {
failureCount++;
std::cerr << "Compilation failed at iteration " << iteration << "\n";
}
}
void modifySourceVersion(int iteration) {
// Modifier TestModule.cpp pour changer version
std::string sourcePath = buildDir + "/../tests/modules/TestModule.cpp";
std::ifstream input(sourcePath);
std::string content((std::istreambuf_iterator<char>(input)), std::istreambuf_iterator<char>());
input.close();
// Remplacer version
std::regex versionRegex(R"(moduleVersion = "v[0-9]+")");
std::string newVersion = "moduleVersion = \"v" + std::to_string(iteration) + "\"";
content = std::regex_replace(content, versionRegex, newVersion);
std::ofstream output(sourcePath);
output << content;
}
};
```
### Test Principal
```cpp
// test_04_race_condition.cpp
#include "helpers/AutoCompiler.h"
#include "helpers/TestMetrics.h"
#include "helpers/TestReporter.h"
#include <atomic>
#include <thread>
int main() {
TestReporter reporter("Race Condition Hunter");
TestMetrics metrics;
const int TOTAL_COMPILATIONS = 1000;
const int COMPILE_INTERVAL_MS = 300;
std::cout << "================================================================================\n";
std::cout << "RACE CONDITION HUNTER: " << TOTAL_COMPILATIONS << " compilations\n";
std::cout << "================================================================================\n\n";
// === SETUP ===
DebugEngine engine;
engine.loadModule("TestModule", "build/modules/libTestModule.so");
auto config = createJsonConfig({{"version", "v0"}});
engine.initializeModule("TestModule", config);
// === STATISTIQUES ===
std::atomic<int> reloadAttempts{0};
std::atomic<int> reloadSuccesses{0};
std::atomic<int> reloadFailures{0};
std::atomic<int> corruptedLoads{0};
std::atomic<int> crashes{0};
std::atomic<bool> engineRunning{true};
// === THREAD 1: Auto-Compiler ===
std::cout << "Starting auto-compiler (300ms interval)...\n";
AutoCompiler compiler("TestModule", "build");
compiler.start(TOTAL_COMPILATIONS, COMPILE_INTERVAL_MS);
// === THREAD 2: FileWatcher + Reload ===
std::cout << "Starting FileWatcher...\n";
std::thread watcherThread([&]() {
std::string soPath = "build/modules/libTestModule.so";
std::filesystem::file_time_type lastWriteTime;
try {
lastWriteTime = std::filesystem::last_write_time(soPath);
} catch (...) {
std::cerr << "Failed to get initial file time\n";
return;
}
while (engineRunning && compiler.getCurrentIteration() < TOTAL_COMPILATIONS) {
try {
auto currentWriteTime = std::filesystem::last_write_time(soPath);
if (currentWriteTime != lastWriteTime) {
// FICHIER MODIFIÉ - RELOAD
reloadAttempts++;
std::cout << "[Compilation #" << compiler.getCurrentIteration()
<< "] File changed, triggering reload...\n";
auto reloadStart = std::chrono::high_resolution_clock::now();
try {
// Le ModuleLoader va attendre file stability
engine.reloadModule("TestModule");
auto reloadEnd = std::chrono::high_resolution_clock::now();
float reloadTime = std::chrono::duration<float, std::milli>(reloadEnd - reloadStart).count();
metrics.recordReloadTime(reloadTime);
reloadSuccesses++;
// Vérifier que le module est valide
auto state = engine.getModuleState("TestModule");
auto* jsonNode = dynamic_cast<JsonDataNode*>(state.get());
const auto& stateJson = jsonNode->getJsonData();
std::string version = stateJson["version"];
std::cout << " → Reload OK (" << reloadTime << "ms), version: " << version << "\n";
} catch (const std::exception& e) {
reloadFailures++;
std::cerr << " → Reload FAILED: " << e.what() << "\n";
// Vérifier si c'est un .so corrompu
if (std::string(e.what()).find("Incomplete") != std::string::npos ||
std::string(e.what()).find("dlopen") != std::string::npos) {
corruptedLoads++;
}
}
lastWriteTime = currentWriteTime;
}
} catch (const std::filesystem::filesystem_error& e) {
// Fichier en cours d'écriture, ignore
}
std::this_thread::sleep_for(std::chrono::milliseconds(50));
}
});
// === THREAD 3: Engine Loop ===
std::cout << "Starting engine loop (60 FPS)...\n";
std::thread engineThread([&]() {
int frame = 0;
while (engineRunning && compiler.getCurrentIteration() < TOTAL_COMPILATIONS) {
auto frameStart = std::chrono::high_resolution_clock::now();
try {
engine.update(1.0f / 60.0f);
// Métriques
auto frameEnd = std::chrono::high_resolution_clock::now();
float frameTime = std::chrono::duration<float, std::milli>(frameEnd - frameStart).count();
metrics.recordFPS(1000.0f / frameTime);
if (frame % 60 == 0) {
metrics.recordMemoryUsage(getCurrentMemoryUsage());
}
} catch (const std::exception& e) {
crashes++;
std::cerr << "[Frame " << frame << "] ENGINE CRASH: " << e.what() << "\n";
// Continue malgré le crash (test robustesse)
}
frame++;
// Sleep pour maintenir 60 FPS
auto frameEnd = std::chrono::high_resolution_clock::now();
auto elapsed = std::chrono::duration<float, std::milli>(frameEnd - frameStart).count();
int sleepMs = std::max(0, static_cast<int>(16.67f - elapsed));
std::this_thread::sleep_for(std::chrono::milliseconds(sleepMs));
}
});
// === ATTENDRE FIN ===
std::cout << "\nRunning test...\n";
// Progress monitoring
while (compiler.getCurrentIteration() < TOTAL_COMPILATIONS) {
std::this_thread::sleep_for(std::chrono::seconds(10));
int progress = (compiler.getCurrentIteration() * 100) / TOTAL_COMPILATIONS;
std::cout << "Progress: " << progress << "% ("
<< compiler.getCurrentIteration() << "/" << TOTAL_COMPILATIONS << " compilations)\n";
std::cout << " Reloads: " << reloadSuccesses << " OK, " << reloadFailures << " FAIL\n";
std::cout << " Corrupted loads: " << corruptedLoads << "\n";
std::cout << " Crashes: " << crashes << "\n\n";
}
// Stop tous les threads
engineRunning = false;
compiler.stop();
watcherThread.join();
engineThread.join();
std::cout << "\nAll threads stopped.\n\n";
// === VÉRIFICATIONS FINALES ===
int compileSuccesses = compiler.getSuccessCount();
int compileFailures = compiler.getFailureCount();
float compileSuccessRate = (compileSuccesses * 100.0f) / TOTAL_COMPILATIONS;
float reloadSuccessRate = (reloadAttempts > 0) ? (reloadSuccesses * 100.0f / reloadAttempts) : 100.0f;
// Assertions
ASSERT_GT(compileSuccessRate, 95.0f, "Compile success rate should be > 95%");
reporter.addMetric("compile_success_rate_percent", compileSuccessRate);
ASSERT_EQ(corruptedLoads, 0, "Should have 0 corrupted loads (file stability check should prevent this)");
reporter.addMetric("corrupted_loads", corruptedLoads);
ASSERT_EQ(crashes, 0, "Should have 0 crashes");
reporter.addMetric("crashes", crashes);
// Si on a des reloads, vérifier le success rate
if (reloadAttempts > 0) {
ASSERT_GT(reloadSuccessRate, 99.0f, "Reload success rate should be > 99%");
}
reporter.addMetric("reload_success_rate_percent", reloadSuccessRate);
// Vérifier que file stability check a fonctionné (temps moyen > 0)
float avgReloadTime = metrics.getReloadTimeAvg();
ASSERT_GT(avgReloadTime, 100.0f, "Avg reload time should be > 100ms (file stability wait)");
reporter.addMetric("reload_time_avg_ms", avgReloadTime);
reporter.addMetric("total_compilations", TOTAL_COMPILATIONS);
reporter.addMetric("compile_successes", compileSuccesses);
reporter.addMetric("compile_failures", compileFailures);
reporter.addMetric("reload_attempts", static_cast<int>(reloadAttempts));
reporter.addMetric("reload_successes", static_cast<int>(reloadSuccesses));
reporter.addMetric("reload_failures", static_cast<int>(reloadFailures));
// === RAPPORT FINAL ===
std::cout << "================================================================================\n";
std::cout << "RACE CONDITION HUNTER SUMMARY\n";
std::cout << "================================================================================\n";
std::cout << "Compilations:\n";
std::cout << " Total: " << TOTAL_COMPILATIONS << "\n";
std::cout << " Successes: " << compileSuccesses << " (" << compileSuccessRate << "%)\n";
std::cout << " Failures: " << compileFailures << "\n\n";
std::cout << "Reloads:\n";
std::cout << " Attempts: " << reloadAttempts << "\n";
std::cout << " Successes: " << reloadSuccesses << " (" << reloadSuccessRate << "%)\n";
std::cout << " Failures: " << reloadFailures << "\n";
std::cout << " Corrupted: " << corruptedLoads << "\n\n";
std::cout << "Stability:\n";
std::cout << " Crashes: " << crashes << "\n";
std::cout << " Reload avg: " << avgReloadTime << "ms\n";
std::cout << "================================================================================\n\n";
reporter.printFinalReport();
return reporter.getExitCode();
}
```
---
## 📊 Métriques Collectées
| Métrique | Description | Seuil |
|----------|-------------|-------|
| **compile_success_rate_percent** | % de compilations réussies | > 95% |
| **reload_success_rate_percent** | % de reloads réussis | > 99% |
| **corrupted_loads** | Nombre de .so corrompus chargés | 0 |
| **crashes** | Nombre de crashes engine | 0 |
| **reload_time_avg_ms** | Temps moyen de reload | > 100ms (prouve que file stability fonctionne) |
| **reload_attempts** | Nombre de tentatives de reload | N/A (info) |
---
## ✅ Critères de Succès
### MUST PASS
1. ✅ Compile success rate > 95%
2. ✅ Corrupted loads = 0 (file stability check marche)
3. ✅ Crashes = 0
4. ✅ Reload success rate > 99%
5. ✅ Reload time avg > 100ms (prouve attente file stability)
### NICE TO HAVE
1. ✅ Compile success rate = 100%
2. ✅ Reload success rate = 100%
3. ✅ Reload time avg < 600ms (efficace malgré stability check)
---
## 🔧 Détection de Corruptions
### Dans ModuleLoader::loadModule()
```cpp
// DÉJÀ IMPLÉMENTÉ - Vérification
auto origSize = std::filesystem::file_size(path);
auto copiedSize = std::filesystem::file_size(tempPath);
if (copiedSize != origSize) {
logger->error("❌ Incomplete copy: orig={} bytes, copied={} bytes", origSize, copiedSize);
throw std::runtime_error("Incomplete file copy detected - CORRUPTED");
}
// Tentative dlopen
void* handle = dlopen(tempPath.c_str(), RTLD_NOW | RTLD_LOCAL);
if (!handle) {
logger->error("❌ dlopen failed: {}", dlerror());
throw std::runtime_error(std::string("Failed to load module: ") + dlerror());
}
```
---
## 🐛 Cas d'Erreur Attendus
| Erreur | Cause | Comportement attendu |
|--------|-------|---------------------|
| Corrupted .so loaded | File stability check raté | FAIL - augmenter stableRequired |
| Reload failure | dlopen pendant write | RETRY - file stability devrait éviter |
| Engine crash | Race dans dlopen/dlclose | FAIL - ajouter mutex |
| High reload time variance | Compilation variable | OK - tant que P99 < seuil |
---
## 📝 Output Attendu
```
================================================================================
RACE CONDITION HUNTER: 1000 compilations
================================================================================
Starting auto-compiler (300ms interval)...
Starting FileWatcher...
Starting engine loop (60 FPS)...
Running test...
[Compilation #3] File changed, triggering reload...
→ Reload OK (487ms), version: v3
[Compilation #7] File changed, triggering reload...
→ Reload OK (523ms), version: v7
Progress: 10% (100/1000 compilations)
Reloads: 98 OK, 0 FAIL
Corrupted loads: 0
Crashes: 0
Progress: 20% (200/1000 compilations)
Reloads: 195 OK, 2 FAIL
Corrupted loads: 0
Crashes: 0
...
Progress: 100% (1000/1000 compilations)
Reloads: 987 OK, 5 FAIL
Corrupted loads: 0
Crashes: 0
All threads stopped.
================================================================================
RACE CONDITION HUNTER SUMMARY
================================================================================
Compilations:
Total: 1000
Successes: 998 (99.8%)
Failures: 2
Reloads:
Attempts: 992
Successes: 987 (99.5%)
Failures: 5
Corrupted: 0
Stability:
Crashes: 0
Reload avg: 505ms
================================================================================
METRICS
================================================================================
Compile success: 99.8% (threshold: > 95%) ✓
Reload success: 99.5% (threshold: > 99%) ✓
Corrupted loads: 0 (threshold: 0) ✓
Crashes: 0 (threshold: 0) ✓
Reload time avg: 505ms (threshold: > 100ms) ✓
Result: ✅ PASSED
================================================================================
```
---
## 📅 Planning
**Jour 1 (4h):**
- Implémenter AutoCompiler helper
- Source modification automatique (version bump)
**Jour 2 (4h):**
- Implémenter test_04_race_condition.cpp
- Threading (compiler, watcher, engine)
- Synchronisation + safety
- Debug + validation
---
**Prochaine étape**: `scenario_05_multimodule.md`

466
planTI/scenario_05_multimodule.md Normal file
View File

@ -0,0 +1,466 @@
# Scénario 5: Multi-Module Orchestration
**Priorité**: ⭐⭐ IMPORTANT
**Phase**: 2 (SHOULD HAVE)
**Durée estimée**: ~2 minutes
**Effort implémentation**: ~4-6 heures
---
## 🎯 Objectif
Valider que le hot-reload d'un module spécifique n'affecte pas les autres modules:
- Isolation complète entre modules
- Ordre d'exécution préservé
- State non corrompu dans modules non-reloadés
- Communication inter-modules fonctionnelle (si applicable)
**Critique pour systèmes multi-modules en production.**
---
## 📋 Description
### Setup
Charger 3 modules avec dépendances:
1. **MapModule**: Grille 100x100, pas de dépendance
2. **TankModule**: Dépend de MapModule (positions valides)
3. **ProductionModule**: Spawne des tanks, dépend de TankModule
### Scénario
1. Exécuter pendant 30 secondes avec les 3 modules
2. Hot-reload **ProductionModule** uniquement à t=15s
3. Vérifier que:
- MapModule non affecté (state identique)
- TankModule non affecté (tanks toujours présents)
- ProductionModule rechargé (version mise à jour)
- Ordre d'exécution toujours: Map → Tank → Production
---
## 🏗️ Implémentation
### Module Dependencies
```cpp
// MapModule.h
class MapModule : public IModule {
public:
bool isPositionValid(float x, float y) const {
int ix = static_cast<int>(x);
int iy = static_cast<int>(y);
return ix >= 0 && ix < mapSize && iy >= 0 && iy < mapSize;
}
void process(float deltaTime) override {
// Update fog of war, etc.
frameCount++;
}
private:
int mapSize = 100;
int frameCount = 0;
};
// TankModule.h
class TankModule : public IModule {
public:
void setMapModule(MapModule* map) { mapModule = map; }
void process(float deltaTime) override {
for (auto& tank : tanks) {
// Vérifier position valide via MapModule
if (mapModule && !mapModule->isPositionValid(tank.x, tank.y)) {
// Correction position
tank.x = std::clamp(tank.x, 0.0f, 99.0f);
tank.y = std::clamp(tank.y, 0.0f, 99.0f);
}
// Update position
tank.x += tank.vx * deltaTime;
tank.y += tank.vy * deltaTime;
}
frameCount++;
}
int getTankCount() const { return tanks.size(); }
private:
std::vector<Tank> tanks;
MapModule* mapModule = nullptr;
int frameCount = 0;
};
// ProductionModule.h
class ProductionModule : public IModule {
public:
void setTankModule(TankModule* tanks) { tankModule = tanks; }
void process(float deltaTime) override {
timeSinceLastSpawn += deltaTime;
if (timeSinceLastSpawn >= 1.0f) {
// Notifier TankModule de spawner un tank
// (Simplification: on log juste ici)
spawned++;
timeSinceLastSpawn -= 1.0f;
logger->debug("Spawned tank #{}", spawned);
}
frameCount++;
}
private:
TankModule* tankModule = nullptr;
int spawned = 0;
float timeSinceLastSpawn = 0.0f;
int frameCount = 0;
std::string version = "v1.0";
};
```
### Test Principal
```cpp
// test_05_multimodule.cpp
#include "helpers/TestMetrics.h"
#include "helpers/TestReporter.h"
int main() {
TestReporter reporter("Multi-Module Orchestration");
TestMetrics metrics;
std::cout << "================================================================================\n";
std::cout << "MULTI-MODULE ORCHESTRATION TEST\n";
std::cout << "================================================================================\n\n";
// === SETUP - Charger 3 modules ===
DebugEngine engine;
engine.loadModule("MapModule", "build/modules/libMapModule.so");
engine.loadModule("TankModule", "build/modules/libTankModule.so");
engine.loadModule("ProductionModule", "build/modules/libProductionModule.so");
auto mapConfig = createJsonConfig({{"mapSize", 100}});
auto tankConfig = createJsonConfig({{"tankCount", 50}});
auto prodConfig = createJsonConfig({{"version", "v1.0"}});
engine.initializeModule("MapModule", mapConfig);
engine.initializeModule("TankModule", tankConfig);
engine.initializeModule("ProductionModule", prodConfig);
std::cout << "Loaded 3 modules: Map, Tank, Production\n";
// === PHASE 1: Pre-Reload (15s) ===
std::cout << "\nPhase 1: Running 15s before reload...\n";
for (int frame = 0; frame < 900; frame++) { // 15s * 60 FPS
engine.update(1.0f / 60.0f);
if (frame % 300 == 0) { // Progress toutes les 5s
std::cout << " Frame " << frame << "/900\n";
}
}
// Snapshot states AVANT reload
auto mapStateBefore = engine.getModuleState("MapModule");
auto tankStateBefore = engine.getModuleState("TankModule");
auto prodStateBefore = engine.getModuleState("ProductionModule");
auto* mapJsonBefore = dynamic_cast<JsonDataNode*>(mapStateBefore.get());
auto* tankJsonBefore = dynamic_cast<JsonDataNode*>(tankStateBefore.get());
auto* prodJsonBefore = dynamic_cast<JsonDataNode*>(prodStateBefore.get());
int mapFramesBefore = mapJsonBefore->getJsonData()["frameCount"];
int tankFramesBefore = tankJsonBefore->getJsonData()["frameCount"];
int tankCountBefore = tankJsonBefore->getJsonData()["tanks"].size();
std::string prodVersionBefore = prodJsonBefore->getJsonData()["version"];
std::cout << "\nState snapshot BEFORE reload:\n";
std::cout << " MapModule frames: " << mapFramesBefore << "\n";
std::cout << " TankModule frames: " << tankFramesBefore << "\n";
std::cout << " TankModule tanks: " << tankCountBefore << "\n";
std::cout << " ProductionModule ver: " << prodVersionBefore << "\n\n";
// === HOT-RELOAD ProductionModule UNIQUEMENT ===
std::cout << "Hot-reloading ProductionModule ONLY...\n";
// Modifier version dans source
modifySourceFile("tests/modules/ProductionModule.cpp", "v1.0", "v2.0 HOT-RELOADED");
// Recompiler
system("cmake --build build --target ProductionModule 2>&1 | grep -v '^\\['");
// Reload
auto reloadStart = std::chrono::high_resolution_clock::now();
engine.reloadModule("ProductionModule");
auto reloadEnd = std::chrono::high_resolution_clock::now();
float reloadTime = std::chrono::duration<float, std::milli>(reloadEnd - reloadStart).count();
metrics.recordReloadTime(reloadTime);
std::cout << " → Reload completed in " << reloadTime << "ms\n\n";
// === VÉRIFICATIONS POST-RELOAD ===
std::cout << "Verifying isolation...\n";
auto mapStateAfter = engine.getModuleState("MapModule");
auto tankStateAfter = engine.getModuleState("TankModule");
auto prodStateAfter = engine.getModuleState("ProductionModule");
auto* mapJsonAfter = dynamic_cast<JsonDataNode*>(mapStateAfter.get());
auto* tankJsonAfter = dynamic_cast<JsonDataNode*>(tankStateAfter.get());
auto* prodJsonAfter = dynamic_cast<JsonDataNode*>(prodStateAfter.get());
int mapFramesAfter = mapJsonAfter->getJsonData()["frameCount"];
int tankFramesAfter = tankJsonAfter->getJsonData()["frameCount"];
int tankCountAfter = tankJsonAfter->getJsonData()["tanks"].size();
std::string prodVersionAfter = prodJsonAfter->getJsonData()["version"];
std::cout << "\nState snapshot AFTER reload:\n";
std::cout << " MapModule frames: " << mapFramesAfter << "\n";
std::cout << " TankModule frames: " << tankFramesAfter << "\n";
std::cout << " TankModule tanks: " << tankCountAfter << "\n";
std::cout << " ProductionModule ver: " << prodVersionAfter << "\n\n";
// Assertions: Isolation
bool mapUnaffected = (mapFramesAfter == mapFramesBefore);
bool tankUnaffected = (tankFramesAfter == tankFramesBefore) && (tankCountAfter == tankCountBefore);
bool prodReloaded = (prodVersionAfter.find("v2.0") != std::string::npos);
ASSERT_TRUE(mapUnaffected, "MapModule should be UNAFFECTED by ProductionModule reload");
reporter.addAssertion("map_unaffected", mapUnaffected);
ASSERT_TRUE(tankUnaffected, "TankModule should be UNAFFECTED by ProductionModule reload");
reporter.addAssertion("tank_unaffected", tankUnaffected);
ASSERT_TRUE(prodReloaded, "ProductionModule should be RELOADED with new version");
reporter.addAssertion("production_reloaded", prodReloaded);
// === PHASE 2: Post-Reload (15s) ===
std::cout << "\nPhase 2: Running 15s after reload...\n";
for (int frame = 0; frame < 900; frame++) { // 15s * 60 FPS
engine.update(1.0f / 60.0f);
if (frame % 300 == 0) {
std::cout << " Frame " << frame << "/900\n";
}
}
// Vérifier ordre d'exécution (détection via logs ou instrumentation)
// Pour l'instant, vérifier que tous les modules ont continué de process
auto mapStateFinal = engine.getModuleState("MapModule");
auto tankStateFinal = engine.getModuleState("TankModule");
auto prodStateFinal = engine.getModuleState("ProductionModule");
auto* mapJsonFinal = dynamic_cast<JsonDataNode*>(mapStateFinal.get());
auto* tankJsonFinal = dynamic_cast<JsonDataNode*>(tankStateFinal.get());
auto* prodJsonFinal = dynamic_cast<JsonDataNode*>(prodStateFinal.get());
int mapFramesFinal = mapJsonFinal->getJsonData()["frameCount"];
int tankFramesFinal = tankJsonFinal->getJsonData()["frameCount"];
int prodFramesFinal = prodJsonFinal->getJsonData()["frameCount"];
// Tous devraient avoir ~1800 frames (30s * 60 FPS)
ASSERT_WITHIN(mapFramesFinal, 1800, 50, "MapModule should have ~1800 frames");
ASSERT_WITHIN(tankFramesFinal, 1800, 50, "TankModule should have ~1800 frames");
ASSERT_WITHIN(prodFramesFinal, 900, 50, "ProductionModule should have ~900 frames (restarted)");
reporter.addMetric("map_frames_final", mapFramesFinal);
reporter.addMetric("tank_frames_final", tankFramesFinal);
reporter.addMetric("prod_frames_final", prodFramesFinal);
// === VÉRIFICATIONS FINALES ===
ASSERT_LT(reloadTime, 1000.0f, "Reload time should be < 1s");
reporter.addMetric("reload_time_ms", reloadTime);
// === RAPPORT FINAL ===
std::cout << "\n";
std::cout << "================================================================================\n";
std::cout << "MULTI-MODULE ORCHESTRATION SUMMARY\n";
std::cout << "================================================================================\n";
std::cout << "Isolation Test:\n";
std::cout << " MapModule unaffected: " << (mapUnaffected ? "✓" : "✗") << "\n";
std::cout << " TankModule unaffected: " << (tankUnaffected ? "✓" : "✗") << "\n";
std::cout << " ProductionModule reloaded: " << (prodReloaded ? "✓" : "✗") << "\n\n";
std::cout << "Final Frame Counts:\n";
std::cout << " MapModule: " << mapFramesFinal << " (~1800 expected)\n";
std::cout << " TankModule: " << tankFramesFinal << " (~1800 expected)\n";
std::cout << " ProductionModule: " << prodFramesFinal << " (~900 expected, restarted)\n\n";
std::cout << "Performance:\n";
std::cout << " Reload time: " << reloadTime << "ms\n";
std::cout << "================================================================================\n\n";
reporter.printFinalReport();
return reporter.getExitCode();
}
```
---
## 📊 Métriques Collectées
| Métrique | Description | Seuil |
|----------|-------------|-------|
| **map_unaffected** | MapModule non affecté par reload | true |
| **tank_unaffected** | TankModule non affecté par reload | true |
| **production_reloaded** | ProductionModule bien rechargé | true |
| **reload_time_ms** | Temps de reload | < 1000ms |
| **map_frames_final** | Frames processées par MapModule | ~1800 |
| **tank_frames_final** | Frames processées par TankModule | ~1800 |
| **prod_frames_final** | Frames processées par ProductionModule | ~900 (reset) |
---
## ✅ Critères de Succès
### MUST PASS
1. ✅ MapModule unaffected (frameCount et state identiques)
2. ✅ TankModule unaffected (tankCount et frameCount identiques)
3. ✅ ProductionModule reloaded (version mise à jour)
4. ✅ Reload time < 1s
5. ✅ Aucun crash
6. ✅ Ordre d'exécution préservé (Map → Tank → Production)
### NICE TO HAVE
1. ✅ Reload time < 500ms
2. ✅ Zero impact sur FPS pendant reload
3. ✅ Communication inter-modules fonctionne après reload
---
## 🔧 Execution Order Verification
### Option 1: Instrumentation dans SequentialModuleSystem
```cpp
void SequentialModuleSystem::update(float deltaTime) {
logger->trace("╔════════════════════════════════════════");
logger->trace("║ UPDATE CYCLE START");
for (const auto& [name, module] : modules) {
logger->trace("║ → Processing: {}", name);
module->process(deltaTime);
}
logger->trace("║ UPDATE CYCLE END");
logger->trace("╚════════════════════════════════════════");
}
```
### Option 2: Vérification via frameCount delta
```cpp
// Si l'ordre est Map → Tank → Production:
// - MapModule devrait voir deltaTime en premier
// - Si on reload Production, Map et Tank ne sont pas affectés
// - frameCount de Map et Tank continue linéairement
// - frameCount de Production redémarre à 0
```
---
## 🐛 Cas d'Erreur Attendus
| Erreur | Cause | Action |
|--------|-------|--------|
| MapModule affected | State mal isolé | FAIL - fix state management |
| TankModule affected | Shared memory corruption | FAIL - fix isolation |
| Production not reloaded | Reload raté | FAIL - check reload logic |
| Execution order changed | Module registry corrupted | FAIL - fix registry |
| Crash during reload | Dependency on unloaded module | FAIL - fix dependencies |
---
## 📝 Output Attendu
```
================================================================================
MULTI-MODULE ORCHESTRATION TEST
================================================================================
Loaded 3 modules: Map, Tank, Production
Phase 1: Running 15s before reload...
Frame 0/900
Frame 300/900
Frame 600/900
State snapshot BEFORE reload:
MapModule frames: 900
TankModule frames: 900
TankModule tanks: 50
ProductionModule ver: v1.0
Hot-reloading ProductionModule ONLY...
→ Reload completed in 412ms
Verifying isolation...
State snapshot AFTER reload:
MapModule frames: 900
TankModule frames: 900
TankModule tanks: 50
ProductionModule ver: v2.0 HOT-RELOADED
Phase 2: Running 15s after reload...
Frame 0/900
Frame 300/900
Frame 600/900
================================================================================
MULTI-MODULE ORCHESTRATION SUMMARY
================================================================================
Isolation Test:
MapModule unaffected: ✓
TankModule unaffected: ✓
ProductionModule reloaded: ✓
Final Frame Counts:
MapModule: 1800 (~1800 expected)
TankModule: 1800 (~1800 expected)
ProductionModule: 900 (~900 expected, restarted)
Performance:
Reload time: 412ms
================================================================================
METRICS
================================================================================
Map unaffected: true ✓
Tank unaffected: true ✓
Production reloaded: true ✓
Reload time: 412ms (threshold: < 1000ms) ✓
ASSERTIONS
================================================================================
✓ map_unaffected
✓ tank_unaffected
✓ production_reloaded
✓ reload_time < 1s
Result: ✅ PASSED
================================================================================
```
---
## 📅 Planning
**Jour 1 (3h):**
- Implémenter MapModule, adapter TankModule avec dépendances
- Implémenter ProductionModule
**Jour 2 (3h):**
- Implémenter test_05_multimodule.cpp
- Instrumentation pour vérifier ordre d'exécution
- Debug + validation
---
**Prochaine étape**: `architecture_tests.md` (détails des helpers communs)

281
planTI/seuils_success.md Normal file
View File

@ -0,0 +1,281 @@
# Seuils de Succès - Critères Pass/Fail
Ce document centralise tous les seuils de succès pour chaque scénario de test.
---
## 🎯 Philosophie des Seuils
### Niveaux de Criticité
- **MUST PASS** ✅: Critères obligatoires. Si un seul échoue → test FAIL
- **SHOULD PASS** ⚠️: Critères recommandés. Si échec → WARNING dans logs
- **NICE TO HAVE** 💡: Critères optimaux. Si échec → INFO dans logs
### Rationale
Les seuils sont définis en fonction de:
1. **Production readiness**: Capacité à tourner en prod 24/7
2. **User experience**: Impact sur la fluidité (60 FPS = 16.67ms/frame)
3. **Resource constraints**: Memory, CPU, file descriptors
4. **Industry standards**: Temps de reload acceptable, uptime
---
## 📊 Scénario 1: Production Hot-Reload
### MUST PASS ✅
| Métrique | Seuil | Justification |
|----------|-------|---------------|
| `reload_time_ms` | **< 1000ms** | Reload > 1s = freeze visible pour l'utilisateur |
| `memory_growth_mb` | **< 5MB** | Croissance mémoire significative = leak probable |
| `fps_min` | **> 30** | < 30 FPS = jeu injouable |
| `tank_count_preserved` | **50/50 (100%)** | Perte d'entités = bug critique |
| `positions_preserved` | **100%** | Positions incorrectes = désync gameplay |
| `no_crashes` | **true** | Crash = inacceptable |
### SHOULD PASS ⚠️
| Métrique | Seuil | Justification |
|----------|-------|---------------|
| `reload_time_ms` | **< 500ms** | Reload plus rapide = meilleure UX |
| `fps_min` | **> 50** | > 50 FPS = expérience très fluide |
### NICE TO HAVE 💡
| Métrique | Seuil | Justification |
|----------|-------|---------------|
| `memory_growth_mb` | **< 1MB** | Memory growth minimal = système quasi-parfait |
| `reload_time_ms` | **< 300ms** | Reload imperceptible |
---
## 📊 Scénario 2: Chaos Monkey
### MUST PASS ✅
| Métrique | Seuil | Justification |
|----------|-------|---------------|
| `engine_alive` | **true** | Engine mort = test fail total |
| `no_deadlocks` | **true** | Deadlock = système bloqué |
| `recovery_rate_percent` | **> 95%** | Recovery < 95% = système fragile |
| `memory_growth_mb` | **< 10MB** | 5 min * 2MB/min = acceptable |
### SHOULD PASS ⚠️
| Métrique | Seuil | Justification |
|----------|-------|---------------|
| `recovery_rate_percent` | **= 100%** | Recovery parfaite = robustesse optimale |
| `memory_growth_mb` | **< 5MB** | Quasi stable même avec chaos |
### NICE TO HAVE 💡
| Métrique | Seuil | Justification |
|----------|-------|---------------|
| `reload_time_avg_ms` | **< 500ms** | Reload rapide même pendant chaos |
---
## 📊 Scénario 3: Stress Test (10 minutes)
### MUST PASS ✅
| Métrique | Seuil | Justification |
|----------|-------|---------------|
| `memory_growth_mb` | **< 20MB** | 10 min → 2MB/min = acceptable |
| `fd_leak` | **= 0** | Leak FD = crash système après N heures |
| `fps_min` | **> 30** | Minimum acceptable pour gameplay |
| `reload_time_p99_ms` | **< 1000ms** | P99 > 1s = dégradation visible |
| `cpu_stddev_percent` | **< 10%** | Stabilité CPU = pas de busy loop |
| `no_crashes` | **true** | Crash = fail |
### SHOULD PASS ⚠️
| Métrique | Seuil | Justification |
|----------|-------|---------------|
| `memory_growth_mb` | **< 10MB** | Très stable |
| `reload_time_p99_ms` | **< 750ms** | Excellent |
### NICE TO HAVE 💡
| Métrique | Seuil | Justification |
|----------|-------|---------------|
| `memory_growth_mb` | **< 5MB** | Quasi-parfait |
| `fps_min` | **> 50** | Très fluide |
---
## 📊 Scénario 3: Stress Test (1 heure - Nightly)
### MUST PASS ✅
| Métrique | Seuil | Justification |
|----------|-------|---------------|
| `memory_growth_mb` | **< 100MB** | 1h → ~1.5MB/min = acceptable |
| `fd_leak` | **= 0** | Critique |
| `fps_min` | **> 30** | Minimum acceptable |
| `reload_time_p99_ms` | **< 1000ms** | Pas de dégradation sur durée |
| `cpu_stddev_percent` | **< 10%** | Stabilité |
| `no_crashes` | **true** | Critique |
### SHOULD PASS ⚠️
| Métrique | Seuil | Justification |
|----------|-------|---------------|
| `memory_growth_mb` | **< 50MB** | Très bon |
| `reload_time_p99_ms` | **< 750ms** | Excellent |
---
## 📊 Scénario 4: Race Condition Hunter
### MUST PASS ✅
| Métrique | Seuil | Justification |
|----------|-------|---------------|
| `compile_success_rate_percent` | **> 95%** | Quelques échecs compilation OK (disk IO, etc.) |
| `reload_success_rate_percent` | **> 99%** | Presque tous les reloads doivent marcher |
| `corrupted_loads` | **= 0** | .so corrompu = file stability check raté |
| `crashes` | **= 0** | Race condition non gérée = crash |
| `reload_time_avg_ms` | **> 100ms** | Prouve que file stability check fonctionne (attend ~500ms) |
### SHOULD PASS ⚠️
| Métrique | Seuil | Justification |
|----------|-------|---------------|
| `compile_success_rate_percent` | **= 100%** | Compilations toujours OK = environnement stable |
| `reload_success_rate_percent` | **= 100%** | Parfait |
### NICE TO HAVE 💡
| Métrique | Seuil | Justification |
|----------|-------|---------------|
| `reload_time_avg_ms` | **< 600ms** | Rapide malgré file stability |
---
## 📊 Scénario 5: Multi-Module Orchestration
### MUST PASS ✅
| Métrique | Seuil | Justification |
|----------|-------|---------------|
| `map_unaffected` | **true** | Isolation critique |
| `tank_unaffected` | **true** | Isolation critique |
| `production_reloaded` | **true** | Reload doit marcher |
| `reload_time_ms` | **< 1000ms** | Standard |
| `no_crashes` | **true** | Critique |
| `execution_order_preserved` | **true** | Ordre critique pour dépendances |
### SHOULD PASS ⚠️
| Métrique | Seuil | Justification |
|----------|-------|---------------|
| `reload_time_ms` | **< 500ms** | Bon |
### NICE TO HAVE 💡
| Métrique | Seuil | Justification |
|----------|-------|---------------|
| `zero_fps_impact` | **true** | FPS identique avant/pendant/après reload |
---
## 🔧 Implémentation dans les Tests
### Pattern de Vérification
```cpp
// Dans chaque test
TestReporter reporter("Scenario Name");
// MUST PASS assertions
ASSERT_LT(reloadTime, 1000.0f, "Reload time MUST be < 1000ms");
reporter.addAssertion("reload_time_ok", reloadTime < 1000.0f);
// SHOULD PASS (warning only)
if (reloadTime >= 500.0f) {
std::cout << "⚠️ WARNING: Reload time should be < 500ms (got " << reloadTime << "ms)\n";
}
// NICE TO HAVE (info only)
if (reloadTime < 300.0f) {
std::cout << "💡 EXCELLENT: Reload time < 300ms\n";
}
// Exit code basé sur MUST PASS uniquement
return reporter.getExitCode(); // 0 si tous MUST PASS OK, 1 sinon
```
### TestReporter Extension
```cpp
// Ajouter dans TestReporter
enum class AssertionLevel {
MUST_PASS,
SHOULD_PASS,
NICE_TO_HAVE
};
void addAssertion(const std::string& name, bool passed, AssertionLevel level);
int getExitCode() const {
// Fail si au moins un MUST_PASS échoue
for (const auto& [name, passed, level] : assertions) {
if (level == AssertionLevel::MUST_PASS && !passed) {
return 1;
}
}
return 0;
}
```
---
## 📋 Tableau Récapitulatif - MUST PASS
| Scénario | Métriques Critiques | Valeurs |
|----------|---------------------|---------|
| **Production Hot-Reload** | reload_time, memory_growth, fps_min, state_preservation | < 1s, < 5MB, > 30, 100% |
| **Chaos Monkey** | engine_alive, recovery_rate, memory_growth | true, > 95%, < 10MB |
| **Stress Test (10min)** | memory_growth, fd_leak, fps_min, reload_p99 | < 20MB, 0, > 30, < 1s |
| **Stress Test (1h)** | memory_growth, fd_leak, fps_min, reload_p99 | < 100MB, 0, > 30, < 1s |
| **Race Condition** | corrupted_loads, crashes, reload_success | 0, 0, > 99% |
| **Multi-Module** | isolation, reload_ok, execution_order | 100%, true, preserved |
---
## 🎯 Validation Globale
### Pour que la suite de tests PASSE:
✅ **TOUS** les scénarios Phase 1 (1-2-3) doivent PASSER leurs MUST PASS
✅ **Au moins 80%** des scénarios Phase 2 (4-5) doivent PASSER leurs MUST PASS
### Pour déclarer le système "Production Ready":
✅ Tous les scénarios MUST PASS
✅ Au moins 70% des SHOULD PASS
✅ Aucun crash dans aucun scénario
✅ Stress test 1h (nightly) PASSE
---
## 📝 Révision des Seuils
Les seuils peuvent être ajustés après analyse des résultats initiaux si:
1. Hardware différent (plus lent) justifie seuils plus permissifs
2. Optimisations permettent seuils plus stricts
3. Nouvelles fonctionnalités changent les contraintes
**Process de révision**:
1. Documenter la justification dans ce fichier
2. Mettre à jour les scénarios correspondants
3. Re-run tous les tests avec nouveaux seuils
4. Commit changes avec message clair
---
**Dernière mise à jour**: 2025-11-13
**Version des seuils**: 1.0

163
src/DebugEngine.cpp
View File

@ -2,9 +2,11 @@
#include <grove/JsonDataNode.h> #include <grove/JsonDataNode.h>
#include <grove/JsonDataValue.h> #include <grove/JsonDataValue.h>
#include <grove/ModuleSystemFactory.h> #include <grove/ModuleSystemFactory.h>
#include <grove/SequentialModuleSystem.h>
#include <nlohmann/json.hpp> #include <nlohmann/json.hpp>
#include <fstream> #include <fstream>
#include <filesystem> #include <filesystem>
#include <sstream>
#include <spdlog/sinks/stdout_color_sinks.h> #include <spdlog/sinks/stdout_color_sinks.h>
#include <spdlog/sinks/basic_file_sink.h> #include <spdlog/sinks/basic_file_sink.h>
@ -17,7 +19,7 @@ DebugEngine::DebugEngine() {
auto console_sink = std::make_shared<spdlog::sinks::stdout_color_sink_mt>(); auto console_sink = std::make_shared<spdlog::sinks::stdout_color_sink_mt>();
auto file_sink = std::make_shared<spdlog::sinks::basic_file_sink_mt>("logs/debug_engine.log", true); auto file_sink = std::make_shared<spdlog::sinks::basic_file_sink_mt>("logs/debug_engine.log", true);
console_sink->set_level(spdlog::level::debug); console_sink->set_level(spdlog::level::trace); // FULL VERBOSE MODE
file_sink->set_level(spdlog::level::trace); file_sink->set_level(spdlog::level::trace);
logger = std::make_shared<spdlog::logger>("DebugEngine", logger = std::make_shared<spdlog::logger>("DebugEngine",
@ -85,11 +87,8 @@ void DebugEngine::run() {
float deltaTime = calculateDeltaTime(); float deltaTime = calculateDeltaTime();
step(deltaTime); step(deltaTime);
// Log every 60 frames (roughly every second at 60fps) // FULL VERBOSE: Log EVERY frame
if (frameCount % 60 == 0) { logger->trace("📊 Frame {}: deltaTime: {:.3f}ms", frameCount, deltaTime * 1000);
logger->debug("📊 Frame {}: Running smoothly, deltaTime: {:.3f}ms",
frameCount, deltaTime * 1000);
}
} }
logger->info("🏁 DebugEngine main loop ended"); logger->info("🏁 DebugEngine main loop ended");
@ -411,7 +410,23 @@ void DebugEngine::processModuleSystems(float deltaTime) {
moduleSystems[i]->processModules(deltaTime); moduleSystems[i]->processModules(deltaTime);
} catch (const std::exception& e) { } catch (const std::exception& e) {
logger->error("❌ Error processing module '{}': {}", moduleNames[i], e.what()); logger->error("❌ Module '{}' crashed: {}", moduleNames[i], e.what());
logger->error("🔍 Frame: {}, deltaTime: {:.3f}ms", frameCount, deltaTime * 1000);
// Automatic recovery attempt
try {
logger->info("🔄 Attempting automatic recovery for module '{}'...", moduleNames[i]);
// Reload the module (will preserve state if possible)
reloadModule(moduleNames[i]);
logger->info("✅ Recovery successful for module '{}'", moduleNames[i]);
} catch (const std::exception& recoveryError) {
logger->critical("❌ Recovery failed for module '{}': {}", moduleNames[i], recoveryError.what());
logger->critical("⚠️ Module '{}' is now in a failed state and will be skipped", moduleNames[i]);
// Continue processing other modules - don't crash the entire engine
}
} }
} }
} }
@ -545,33 +560,47 @@ void DebugEngine::reloadModule(const std::string& name) {
auto& moduleSystem = moduleSystems[index]; auto& moduleSystem = moduleSystems[index];
auto& loader = moduleLoaders[index]; auto& loader = moduleLoaders[index];
// Step 1: Extract module from system (SequentialModuleSystem has extractModule) // Step 1: Extract module from system
logger->debug("📤 Step 1/3: Extracting module from system"); logger->debug("📤 Step 1/4: Extracting module from system");
// We need to cast to SequentialModuleSystem to access extractModule
// For now, we'll work around this by getting the current module state
// Step 2: Reload via loader (handles state preservation) // Try to cast to SequentialModuleSystem to access extractModule()
logger->debug("🔄 Step 2/3: Reloading module via loader"); auto* seqSystem = dynamic_cast<SequentialModuleSystem*>(moduleSystem.get());
if (!seqSystem) {
logger->error("❌ Hot-reload only supported for SequentialModuleSystem currently");
throw std::runtime_error("Hot-reload not supported for this module system type");
}
// For SequentialModuleSystem, we need to extract the module first auto currentModule = seqSystem->extractModule();
// This is a limitation of the current IModuleSystem interface if (!currentModule) {
// We'll need to get the state via queryModule as a workaround logger->error("❌ Failed to extract module from system");
throw std::runtime_error("Failed to extract module");
}
nlohmann::json queryInput = {{"command", "getState"}}; logger->debug("✅ Module extracted successfully");
auto queryData = std::make_unique<JsonDataNode>("query", queryInput);
auto currentState = moduleSystem->queryModule(name, *queryData);
// Unload and reload the .so // Step 2: Wait for clean state (module idle + no pending tasks)
std::string modulePath = loader->getLoadedPath(); logger->debug("⏳ Step 2/5: Waiting for clean state");
loader->unload(); bool cleanState = loader->waitForCleanState(currentModule.get(), seqSystem, 5.0f);
auto newModule = loader->load(modulePath, name); if (!cleanState) {
logger->error("❌ Module did not reach clean state within timeout");
throw std::runtime_error("Hot-reload timeout - module not idle or has pending tasks");
}
logger->debug("✅ Clean state reached");
// Restore state // Step 3: Get current state
newModule->setState(*currentState); logger->debug("💾 Step 3/5: Extracting module state");
auto currentState = currentModule->getState();
logger->debug("✅ State extracted successfully");
// Step 3: Register new module with system // Step 4: Reload module via loader
logger->debug("🔗 Step 3/3: Registering new module with system"); logger->debug("🔄 Step 4/5: Reloading .so file");
auto newModule = loader->reload(std::move(currentModule));
logger->debug("✅ Module reloaded successfully");
// Step 5: Register new module back with system
logger->debug("🔗 Step 5/5: Registering new module with system");
moduleSystem->registerModule(name, std::move(newModule)); moduleSystem->registerModule(name, std::move(newModule));
logger->debug("✅ Module registered successfully");
auto reloadEndTime = std::chrono::high_resolution_clock::now(); auto reloadEndTime = std::chrono::high_resolution_clock::now();
float reloadTime = std::chrono::duration<float, std::milli>(reloadEndTime - reloadStartTime).count(); float reloadTime = std::chrono::duration<float, std::milli>(reloadEndTime - reloadStartTime).count();
@ -584,4 +613,84 @@ void DebugEngine::reloadModule(const std::string& name) {
} }
} }
void DebugEngine::dumpModuleState(const std::string& name) {
logger->info("╔══════════════════════════════════════════════════════════════");
logger->info("║ 📊 STATE DUMP: {}", name);
logger->info("╠══════════════════════════════════════════════════════════════");
try {
// Find module index
auto it = std::find(moduleNames.begin(), moduleNames.end(), name);
if (it == moduleNames.end()) {
logger->error("║ ❌ Module '{}' not found", name);
logger->info("╚══════════════════════════════════════════════════════════════");
return;
}
size_t index = std::distance(moduleNames.begin(), it);
auto& moduleSystem = moduleSystems[index];
// Try to cast to SequentialModuleSystem to access module
auto* seqSystem = dynamic_cast<SequentialModuleSystem*>(moduleSystem.get());
if (!seqSystem) {
logger->warn("║ ⚠️ State dump only supported for SequentialModuleSystem currently");
logger->info("╚══════════════════════════════════════════════════════════════");
return;
}
// Extract module temporarily
auto module = seqSystem->extractModule();
if (!module) {
logger->error("║ ❌ Failed to extract module");
logger->info("╚══════════════════════════════════════════════════════════════");
return;
}
// Get state
auto state = module->getState();
// Cast to JsonDataNode to access JSON
auto* jsonNode = dynamic_cast<JsonDataNode*>(state.get());
if (!jsonNode) {
logger->warn("║ ⚠️ State is not JsonDataNode, cannot dump as JSON");
moduleSystem->registerModule(name, std::move(module));
logger->info("╚══════════════════════════════════════════════════════════════");
return;
}
// Convert to JSON and pretty print
const auto& jsonState = jsonNode->getJsonData();
std::string prettyJson = jsonState.dump(2); // 2 spaces indentation
// Split into lines and print with border
std::istringstream stream(prettyJson);
std::string line;
while (std::getline(stream, line)) {
logger->info("║ {}", line);
}
// Re-register module (we only borrowed it)
moduleSystem->registerModule(name, std::move(module));
logger->info("╚══════════════════════════════════════════════════════════════");
} catch (const std::exception& e) {
logger->error("║ ❌ Error dumping state: {}", e.what());
logger->info("╚══════════════════════════════════════════════════════════════");
}
}
void DebugEngine::dumpAllModulesState() {
logger->info("╔══════════════════════════════════════════════════════════════");
logger->info("║ 📊 DUMPING ALL MODULE STATES ({} modules)", moduleNames.size());
logger->info("╚══════════════════════════════════════════════════════════════");
for (const auto& moduleName : moduleNames) {
dumpModuleState(moduleName);
logger->info(""); // Blank line between modules
}
logger->info("✅ All module states dumped");
}
} // namespace grove } // namespace grove

159
src/ModuleLoader.cpp
View File

@ -1,5 +1,11 @@
#include <grove/ModuleLoader.h> #include <grove/ModuleLoader.h>
#include <grove/IModuleSystem.h>
#include <chrono> #include <chrono>
#include <cstdio>
#include <cstring>
#include <unistd.h>
#include <filesystem>
#include <thread>
#include <spdlog/sinks/stdout_color_sinks.h> #include <spdlog/sinks/stdout_color_sinks.h>
namespace grove { namespace grove {
@ -24,19 +30,113 @@ ModuleLoader::~ModuleLoader() {
} }
} }
std::unique_ptr<IModule> ModuleLoader::load(const std::string& path, const std::string& name) { std::unique_ptr<IModule> ModuleLoader::load(const std::string& path, const std::string& name, bool isReload) {
logLoadStart(path); logLoadStart(path);
auto loadStartTime = std::chrono::high_resolution_clock::now(); auto loadStartTime = std::chrono::high_resolution_clock::now();
// CRITICAL FIX: Linux caches .so files by path in dlopen
// Even after dlclose, subsequent dlopen of the same path will load from cache
// Solution: Create a temporary copy with unique name for hot-reload scenarios
std::string actualPath = path;
std::string tempPath;
bool usedTempCopy = false;
// If we're reloading, use a temp copy to bypass cache
if (isReload) {
// CRITICAL FIX: Wait for file to be fully written after compilation
// The FileWatcher may detect the change while the compiler is still writing
logger->debug("⏳ Waiting for .so file to be fully written...");
size_t lastSize = 0;
size_t stableCount = 0;
const int maxAttempts = 20; // 1 second max wait (20 * 50ms)
const int stableRequired = 3; // Require 3 consecutive stable readings
for (int i = 0; i < maxAttempts; i++) {
try {
size_t currentSize = std::filesystem::file_size(path);
if (currentSize > 0 && currentSize == lastSize) {
stableCount++;
if (stableCount >= stableRequired) {
logger->debug("✅ File size stable at {} bytes (after {}ms)", currentSize, i * 50);
break;
}
} else {
stableCount = 0; // Reset if size changed
}
lastSize = currentSize;
std::this_thread::sleep_for(std::chrono::milliseconds(50));
} catch (const std::filesystem::filesystem_error& e) {
// File might not exist yet or be locked
logger->debug("⏳ Waiting for file access... ({})", e.what());
std::this_thread::sleep_for(std::chrono::milliseconds(50));
}
}
// Create unique temp filename
char tempTemplate[] = "/tmp/grove_module_XXXXXX.so";
int tempFd = mkstemps(tempTemplate, 3); // 3 for ".so"
if (tempFd == -1) {
logger->warn("⚠️ Failed to create temp file, loading directly (may use cached version)");
} else {
close(tempFd); // Close the fd, we just need the unique name
tempPath = tempTemplate;
// Copy original .so to temp location using std::filesystem
try {
std::filesystem::copy_file(path, tempPath,
std::filesystem::copy_options::overwrite_existing);
// CRITICAL FIX: Verify the copy succeeded completely
auto origSize = std::filesystem::file_size(path);
auto copiedSize = std::filesystem::file_size(tempPath);
if (copiedSize != origSize) {
logger->error("❌ Incomplete copy: orig={} bytes, copied={} bytes", origSize, copiedSize);
unlink(tempPath.c_str());
throw std::runtime_error("Incomplete file copy detected");
}
if (origSize == 0) {
logger->error("❌ Source file is empty!");
unlink(tempPath.c_str());
throw std::runtime_error("Source .so file is empty");
}
actualPath = tempPath;
usedTempCopy = true;
logger->debug("🔄 Using temp copy for hot-reload: {} ({} bytes)", tempPath, copiedSize);
} catch (const std::filesystem::filesystem_error& e) {
logger->warn("⚠️ Failed to copy .so ({}), loading directly (may use cached version)", e.what());
unlink(tempPath.c_str()); // Clean up failed temp file
}
}
}
// Open library with RTLD_NOW (resolve all symbols immediately) // Open library with RTLD_NOW (resolve all symbols immediately)
libraryHandle = dlopen(path.c_str(), RTLD_NOW); libraryHandle = dlopen(actualPath.c_str(), RTLD_NOW);
if (!libraryHandle) { if (!libraryHandle) {
std::string error = dlerror(); std::string error = dlerror();
// Clean up temp file if it was created
if (usedTempCopy) {
unlink(tempPath.c_str());
}
logLoadError(error); logLoadError(error);
throw std::runtime_error("Failed to load module: " + error); throw std::runtime_error("Failed to load module: " + error);
} }
// Store temp path for cleanup later
if (usedTempCopy) {
tempLibraryPath = tempPath;
logger->debug("📝 Stored temp path for cleanup: {}", tempLibraryPath);
}
// Find createModule factory function // Find createModule factory function
createFunc = reinterpret_cast<CreateModuleFunc>(dlsym(libraryHandle, "createModule")); createFunc = reinterpret_cast<CreateModuleFunc>(dlsym(libraryHandle, "createModule"));
if (!createFunc) { if (!createFunc) {
@ -83,6 +183,17 @@ void ModuleLoader::unload() {
logger->error("❌ dlclose failed: {}", error); logger->error("❌ dlclose failed: {}", error);
} }
// Clean up temp file if it was used
if (!tempLibraryPath.empty()) {
logger->debug("🧹 Cleaning up temp file: {}", tempLibraryPath);
if (unlink(tempLibraryPath.c_str()) == 0) {
logger->debug("✅ Temp file deleted");
} else {
logger->warn("⚠️ Failed to delete temp file: {}", tempLibraryPath);
}
tempLibraryPath.clear();
}
libraryHandle = nullptr; libraryHandle = nullptr;
createFunc = nullptr; createFunc = nullptr;
libraryPath.clear(); libraryPath.clear();
@ -91,6 +202,44 @@ void ModuleLoader::unload() {
logUnloadSuccess(); logUnloadSuccess();
} }
bool ModuleLoader::waitForCleanState(IModule* module, IModuleSystem* moduleSystem, float timeoutSeconds) {
logger->info("⏳ Waiting for clean state (timeout: {:.1f}s)", timeoutSeconds);
auto startTime = std::chrono::high_resolution_clock::now();
auto lastLogTime = startTime;
while (true) {
auto currentTime = std::chrono::high_resolution_clock::now();
float elapsed = std::chrono::duration<float>(currentTime - startTime).count();
// Check timeout
if (elapsed >= timeoutSeconds) {
logger->error("❌ Clean state timeout after {:.1f}s", elapsed);
return false;
}
// Check clean state conditions
bool moduleIdle = module->isIdle();
int pendingTasks = moduleSystem->getPendingTaskCount(moduleName);
if (moduleIdle && pendingTasks == 0) {
logger->info("✅ Clean state reached after {:.3f}s", elapsed);
return true;
}
// Log progress every 500ms
float timeSinceLastLog = std::chrono::duration<float>(currentTime - lastLogTime).count();
if (timeSinceLastLog >= 0.5f) {
logger->info("⏳ Waiting... ({:.1f}s) - module idle: {}, pending tasks: {}",
elapsed, moduleIdle, pendingTasks);
lastLogTime = currentTime;
}
// Sleep briefly to avoid busy waiting
std::this_thread::sleep_for(std::chrono::milliseconds(10));
}
}
std::unique_ptr<IModule> ModuleLoader::reload(std::unique_ptr<IModule> currentModule) { std::unique_ptr<IModule> ModuleLoader::reload(std::unique_ptr<IModule> currentModule) {
logger->info("🔄 Hot-reload starting for module '{}'", moduleName); logger->info("🔄 Hot-reload starting for module '{}'", moduleName);
@ -123,9 +272,9 @@ std::unique_ptr<IModule> ModuleLoader::reload(std::unique_ptr<IModule> currentMo
unload(); unload();
logger->debug("✅ Old library unloaded"); logger->debug("✅ Old library unloaded");
// Step 4: Load new library and restore state // Step 4: Load new library and restore state (use temp copy to bypass cache)
logger->debug("📥 Step 4/4: Loading new library"); logger->debug("📥 Step 4/4: Loading new library with cache bypass");
auto newModule = load(pathToReload, nameToReload); auto newModule = load(pathToReload, nameToReload, true); // isReload = true
logger->debug("✅ New library loaded"); logger->debug("✅ New library loaded");
// Step 5: Restore state to new module // Step 5: Restore state to new module

8
src/SequentialModuleSystem.cpp
View File

@ -11,7 +11,7 @@ SequentialModuleSystem::SequentialModuleSystem() {
auto console_sink = std::make_shared<spdlog::sinks::stdout_color_sink_mt>(); auto console_sink = std::make_shared<spdlog::sinks::stdout_color_sink_mt>();
auto file_sink = std::make_shared<spdlog::sinks::basic_file_sink_mt>("logs/sequential_system.log", true); auto file_sink = std::make_shared<spdlog::sinks::basic_file_sink_mt>("logs/sequential_system.log", true);
console_sink->set_level(spdlog::level::debug); console_sink->set_level(spdlog::level::trace); // FULL VERBOSE MODE
file_sink->set_level(spdlog::level::trace); file_sink->set_level(spdlog::level::trace);
logger = std::make_shared<spdlog::logger>("SequentialModuleSystem", logger = std::make_shared<spdlog::logger>("SequentialModuleSystem",
@ -158,6 +158,12 @@ ModuleSystemType SequentialModuleSystem::getType() const {
return ModuleSystemType::SEQUENTIAL; return ModuleSystemType::SEQUENTIAL;
} }
int SequentialModuleSystem::getPendingTaskCount(const std::string& moduleName) const {
// SequentialModuleSystem executes tasks immediately, so never has pending tasks
logger->trace("🔍 Pending task count for '{}': 0 (sequential execution)", moduleName);
return 0;
}
// ITaskScheduler implementation // ITaskScheduler implementation
void SequentialModuleSystem::scheduleTask(const std::string& taskType, std::unique_ptr<IDataNode> taskData) { void SequentialModuleSystem::scheduleTask(const std::string& taskType, std::unique_ptr<IDataNode> taskData) {
logger->debug("⚙️ Task scheduled for immediate execution: '{}'", taskType); logger->debug("⚙️ Task scheduled for immediate execution: '{}'", taskType);

107
tests/CMakeLists.txt
View File

@ -55,3 +55,110 @@ add_custom_command(TARGET test_engine_hotreload POST_BUILD
$<TARGET_FILE_DIR:test_engine_hotreload>/ $<TARGET_FILE_DIR:test_engine_hotreload>/
COMMENT "Copying TestModule.so to engine test directory" COMMENT "Copying TestModule.so to engine test directory"
) )
# ================================================================================
# Integration Tests
# ================================================================================
# Helpers library (partagée par tous les tests)
add_library(test_helpers STATIC
helpers/TestMetrics.cpp
helpers/TestReporter.cpp
helpers/SystemUtils.cpp
)
target_include_directories(test_helpers PUBLIC
${CMAKE_CURRENT_SOURCE_DIR}
)
target_link_libraries(test_helpers PUBLIC
GroveEngine::core
spdlog::spdlog
)
# Set PIC for static library
set_target_properties(test_helpers PROPERTIES POSITION_INDEPENDENT_CODE ON)
# TankModule pour tests d'intégration
add_library(TankModule SHARED
modules/TankModule.cpp
)
target_link_libraries(TankModule PRIVATE
GroveEngine::core
GroveEngine::impl
spdlog::spdlog
)
# Ensure spdlog is compiled with PIC
set_target_properties(spdlog PROPERTIES POSITION_INDEPENDENT_CODE ON)
# Test 01: Production Hot-Reload
add_executable(test_01_production_hotreload
integration/test_01_production_hotreload.cpp
)
target_link_libraries(test_01_production_hotreload PRIVATE
test_helpers
GroveEngine::core
GroveEngine::impl
)
add_dependencies(test_01_production_hotreload TankModule)
# CTest integration
add_test(NAME ProductionHotReload COMMAND test_01_production_hotreload)
# ChaosModule pour tests de robustesse
add_library(ChaosModule SHARED
modules/ChaosModule.cpp
)
target_link_libraries(ChaosModule PRIVATE
GroveEngine::core
GroveEngine::impl
spdlog::spdlog
)
# Test 02: Chaos Monkey
add_executable(test_02_chaos_monkey
integration/test_02_chaos_monkey.cpp
)
target_link_libraries(test_02_chaos_monkey PRIVATE
test_helpers
GroveEngine::core
GroveEngine::impl
)
add_dependencies(test_02_chaos_monkey ChaosModule)
# CTest integration
add_test(NAME ChaosMonkey COMMAND test_02_chaos_monkey)
# StressModule pour tests de stabilité long-terme
add_library(StressModule SHARED
modules/StressModule.cpp
)
target_link_libraries(StressModule PRIVATE
GroveEngine::core
GroveEngine::impl
spdlog::spdlog
)
# Test 03: Stress Test - 10 minutes stability
add_executable(test_03_stress_test
integration/test_03_stress_test.cpp
)
target_link_libraries(test_03_stress_test PRIVATE
test_helpers
GroveEngine::core
GroveEngine::impl
)
add_dependencies(test_03_stress_test StressModule)
# CTest integration
add_test(NAME StressTest COMMAND test_03_stress_test)

49
tests/helpers/SystemUtils.cpp Normal file
View File

@ -0,0 +1,49 @@
#include "SystemUtils.h"
#include <fstream>
#include <string>
#include <dirent.h>
#include <sstream>
namespace grove {
size_t getCurrentMemoryUsage() {
// Linux: /proc/self/status -> VmRSS
std::ifstream file("/proc/self/status");
std::string line;
while (std::getline(file, line)) {
if (line.substr(0, 6) == "VmRSS:") {
std::istringstream iss(line.substr(7));
size_t kb;
iss >> kb;
return kb * 1024; // Convert to bytes
}
}
return 0;
}
int getOpenFileDescriptors() {
// Linux: /proc/self/fd
int count = 0;
DIR* dir = opendir("/proc/self/fd");
if (dir) {
struct dirent* entry;
while ((entry = readdir(dir)) != nullptr) {
count++;
}
closedir(dir);
}
return count - 2; // Exclude . and ..
}
float getCurrentCPUUsage() {
// Simplifié - retourne 0 pour l'instant
// Implémentation complète nécessite tracking du /proc/self/stat
// entre deux lectures (utime + stime delta)
return 0.0f;
}
} // namespace grove

10
tests/helpers/SystemUtils.h Normal file
View File

@ -0,0 +1,10 @@
#pragma once
#include <cstddef>
namespace grove {
size_t getCurrentMemoryUsage();
int getOpenFileDescriptors();
float getCurrentCPUUsage();
} // namespace grove

77
tests/helpers/TestAssertions.h Normal file
View File

@ -0,0 +1,77 @@
#pragma once
#include <iostream>
#include <cstdlib>
#include <cmath>
// Couleurs pour output
#define COLOR_RED "\033[31m"
#define COLOR_GREEN "\033[32m"
#define COLOR_RESET "\033[0m"
#define ASSERT_TRUE(condition, message) \
do { \
if (!(condition)) { \
std::cerr << COLOR_RED << "❌ ASSERTION FAILED: " << message << COLOR_RESET << "\n"; \
std::cerr << " At: " << __FILE__ << ":" << __LINE__ << "\n"; \
std::exit(1); \
} \
} while(0)
#define ASSERT_FALSE(condition, message) \
ASSERT_TRUE(!(condition), message)
#define ASSERT_EQ(actual, expected, message) \
do { \
if ((actual) != (expected)) { \
std::cerr << COLOR_RED << "❌ ASSERTION FAILED: " << message << COLOR_RESET << "\n"; \
std::cerr << " Expected: " << (expected) << "\n"; \
std::cerr << " Actual: " << (actual) << "\n"; \
std::cerr << " At: " << __FILE__ << ":" << __LINE__ << "\n"; \
std::exit(1); \
} \
} while(0)
#define ASSERT_NE(actual, expected, message) \
do { \
if ((actual) == (expected)) { \
std::cerr << COLOR_RED << "❌ ASSERTION FAILED: " << message << COLOR_RESET << "\n"; \
std::cerr << " Should not equal: " << (expected) << "\n"; \
std::cerr << " But got: " << (actual) << "\n"; \
std::cerr << " At: " << __FILE__ << ":" << __LINE__ << "\n"; \
std::exit(1); \
} \
} while(0)
#define ASSERT_LT(value, max, message) \
do { \
if ((value) >= (max)) { \
std::cerr << COLOR_RED << "❌ ASSERTION FAILED: " << message << COLOR_RESET << "\n"; \
std::cerr << " Expected: < " << (max) << "\n"; \
std::cerr << " Actual: " << (value) << "\n"; \
std::cerr << " At: " << __FILE__ << ":" << __LINE__ << "\n"; \
std::exit(1); \
} \
} while(0)
#define ASSERT_GT(value, min, message) \
do { \
if ((value) <= (min)) { \
std::cerr << COLOR_RED << "❌ ASSERTION FAILED: " << message << COLOR_RESET << "\n"; \
std::cerr << " Expected: > " << (min) << "\n"; \
std::cerr << " Actual: " << (value) << "\n"; \
std::cerr << " At: " << __FILE__ << ":" << __LINE__ << "\n"; \
std::exit(1); \
} \
} while(0)
#define ASSERT_WITHIN(actual, expected, tolerance, message) \
do { \
auto diff = std::abs((actual) - (expected)); \
if (diff > (tolerance)) { \
std::cerr << COLOR_RED << "❌ ASSERTION FAILED: " << message << COLOR_RESET << "\n"; \
std::cerr << " Expected: " << (expected) << " ± " << (tolerance) << "\n"; \
std::cerr << " Actual: " << (actual) << " (diff: " << diff << ")\n"; \
std::cerr << " At: " << __FILE__ << ":" << __LINE__ << "\n"; \
std::exit(1); \
} \
} while(0)

170
tests/helpers/TestMetrics.cpp Normal file
View File

@ -0,0 +1,170 @@
#include "TestMetrics.h"
#include <iostream>
#include <iomanip>
namespace grove {
void TestMetrics::recordFPS(float fps) {
fpsValues.push_back(fps);
}
void TestMetrics::recordMemoryUsage(size_t bytes) {
if (!hasInitialMemory) {
initialMemory = bytes;
hasInitialMemory = true;
}
memoryValues.push_back(bytes);
}
void TestMetrics::recordReloadTime(float ms) {
reloadTimes.push_back(ms);
}
void TestMetrics::recordCrash(const std::string& reason) {
crashReasons.push_back(reason);
}
float TestMetrics::getFPSMin() const {
if (fpsValues.empty()) return 0.0f;
return *std::min_element(fpsValues.begin(), fpsValues.end());
}
float TestMetrics::getFPSMax() const {
if (fpsValues.empty()) return 0.0f;
return *std::max_element(fpsValues.begin(), fpsValues.end());
}
float TestMetrics::getFPSAvg() const {
if (fpsValues.empty()) return 0.0f;
return std::accumulate(fpsValues.begin(), fpsValues.end(), 0.0f) / fpsValues.size();
}
float TestMetrics::getFPSStdDev() const {
if (fpsValues.empty()) return 0.0f;
float avg = getFPSAvg();
float variance = 0.0f;
for (float fps : fpsValues) {
variance += std::pow(fps - avg, 2);
}
return std::sqrt(variance / fpsValues.size());
}
float TestMetrics::getFPSMinLast60s() const {
if (fpsValues.empty()) return 0.0f;
// Last 60 seconds = 3600 frames at 60 FPS
size_t startIdx = fpsValues.size() > 3600 ? fpsValues.size() - 3600 : 0;
auto it = std::min_element(fpsValues.begin() + startIdx, fpsValues.end());
return *it;
}
float TestMetrics::getFPSAvgLast60s() const {
if (fpsValues.empty()) return 0.0f;
size_t startIdx = fpsValues.size() > 3600 ? fpsValues.size() - 3600 : 0;
float sum = std::accumulate(fpsValues.begin() + startIdx, fpsValues.end(), 0.0f);
return sum / (fpsValues.size() - startIdx);
}
size_t TestMetrics::getMemoryInitial() const {
return initialMemory;
}
size_t TestMetrics::getMemoryFinal() const {
if (memoryValues.empty()) return 0;
return memoryValues.back();
}
size_t TestMetrics::getMemoryGrowth() const {
if (memoryValues.empty()) return 0;
return memoryValues.back() - initialMemory;
}
size_t TestMetrics::getMemoryPeak() const {
if (memoryValues.empty()) return 0;
return *std::max_element(memoryValues.begin(), memoryValues.end());
}
float TestMetrics::getReloadTimeAvg() const {
if (reloadTimes.empty()) return 0.0f;
return std::accumulate(reloadTimes.begin(), reloadTimes.end(), 0.0f) / reloadTimes.size();
}
float TestMetrics::getReloadTimeMin() const {
if (reloadTimes.empty()) return 0.0f;
return *std::min_element(reloadTimes.begin(), reloadTimes.end());
}
float TestMetrics::getReloadTimeMax() const {
if (reloadTimes.empty()) return 0.0f;
return *std::max_element(reloadTimes.begin(), reloadTimes.end());
}
float TestMetrics::getReloadTimeP99() const {
if (reloadTimes.empty()) return 0.0f;
auto sorted = reloadTimes;
std::sort(sorted.begin(), sorted.end());
size_t p99Index = static_cast<size_t>(sorted.size() * 0.99);
if (p99Index >= sorted.size()) p99Index = sorted.size() - 1;
return sorted[p99Index];
}
int TestMetrics::getReloadCount() const {
return static_cast<int>(reloadTimes.size());
}
int TestMetrics::getCrashCount() const {
return static_cast<int>(crashReasons.size());
}
const std::vector<std::string>& TestMetrics::getCrashReasons() const {
return crashReasons;
}
void TestMetrics::printReport() const {
std::cout << "╔══════════════════════════════════════════════════════════════\n";
std::cout << "║ METRICS REPORT\n";
std::cout << "╠══════════════════════════════════════════════════════════════\n";
if (!fpsValues.empty()) {
std::cout << "║ FPS:\n";
std::cout << "║ Min: " << std::setw(8) << std::fixed << std::setprecision(2) << getFPSMin() << "\n";
std::cout << "║ Avg: " << std::setw(8) << std::fixed << std::setprecision(2) << getFPSAvg() << "\n";
std::cout << "║ Max: " << std::setw(8) << std::fixed << std::setprecision(2) << getFPSMax() << "\n";
std::cout << "║ Std Dev: " << std::setw(8) << std::fixed << std::setprecision(2) << getFPSStdDev() << "\n";
}
if (!memoryValues.empty()) {
std::cout << "║ Memory:\n";
std::cout << "║ Initial: " << std::setw(8) << std::fixed << std::setprecision(2)
<< (initialMemory / 1024.0f / 1024.0f) << " MB\n";
std::cout << "║ Final: " << std::setw(8) << std::fixed << std::setprecision(2)
<< (memoryValues.back() / 1024.0f / 1024.0f) << " MB\n";
std::cout << "║ Peak: " << std::setw(8) << std::fixed << std::setprecision(2)
<< (getMemoryPeak() / 1024.0f / 1024.0f) << " MB\n";
std::cout << "║ Growth: " << std::setw(8) << std::fixed << std::setprecision(2)
<< (getMemoryGrowth() / 1024.0f / 1024.0f) << " MB\n";
}
if (!reloadTimes.empty()) {
std::cout << "║ Reload Times:\n";
std::cout << "║ Count: " << std::setw(8) << reloadTimes.size() << "\n";
std::cout << "║ Avg: " << std::setw(8) << std::fixed << std::setprecision(2)
<< getReloadTimeAvg() << " ms\n";
std::cout << "║ Min: " << std::setw(8) << std::fixed << std::setprecision(2)
<< getReloadTimeMin() << " ms\n";
std::cout << "║ Max: " << std::setw(8) << std::fixed << std::setprecision(2)
<< getReloadTimeMax() << " ms\n";
std::cout << "║ P99: " << std::setw(8) << std::fixed << std::setprecision(2)
<< getReloadTimeP99() << " ms\n";
}
if (!crashReasons.empty()) {
std::cout << "║ Crashes: " << crashReasons.size() << "\n";
for (const auto& reason : crashReasons) {
std::cout << "║ - " << reason << "\n";
}
}
std::cout << "╚══════════════════════════════════════════════════════════════\n";
}
} // namespace grove

56
tests/helpers/TestMetrics.h Normal file
View File

@ -0,0 +1,56 @@
#pragma once
#include <vector>
#include <algorithm>
#include <numeric>
#include <cmath>
#include <string>
namespace grove {
class TestMetrics {
public:
// Enregistrement
void recordFPS(float fps);
void recordMemoryUsage(size_t bytes);
void recordReloadTime(float ms);
void recordCrash(const std::string& reason);
// Getters - FPS
float getFPSMin() const;
float getFPSMax() const;
float getFPSAvg() const;
float getFPSStdDev() const;
float getFPSMinLast60s() const; // Pour stress test
float getFPSAvgLast60s() const;
// Getters - Memory
size_t getMemoryInitial() const;
size_t getMemoryFinal() const;
size_t getMemoryPeak() const;
size_t getMemoryGrowth() const;
// Getters - Reload
float getReloadTimeAvg() const;
float getReloadTimeMin() const;
float getReloadTimeMax() const;
float getReloadTimeP99() const; // Percentile 99
int getReloadCount() const;
// Getters - Crashes
int getCrashCount() const;
const std::vector<std::string>& getCrashReasons() const;
// Rapport
void printReport() const;
private:
std::vector<float> fpsValues;
std::vector<size_t> memoryValues;
std::vector<float> reloadTimes;
std::vector<std::string> crashReasons;
size_t initialMemory = 0;
bool hasInitialMemory = false;
};
} // namespace grove

58
tests/helpers/TestReporter.cpp Normal file
View File

@ -0,0 +1,58 @@
#include "TestReporter.h"
#include <iostream>
namespace grove {
TestReporter::TestReporter(const std::string& name) : scenarioName(name) {}
void TestReporter::addMetric(const std::string& name, float value) {
metrics[name] = value;
}
void TestReporter::addAssertion(const std::string& name, bool passed) {
assertions.push_back({name, passed});
}
void TestReporter::printFinalReport() const {
std::cout << "\n";
std::cout << "════════════════════════════════════════════════════════════════\n";
std::cout << "FINAL REPORT: " << scenarioName << "\n";
std::cout << "════════════════════════════════════════════════════════════════\n\n";
// Metrics
if (!metrics.empty()) {
std::cout << "Metrics:\n";
for (const auto& [name, value] : metrics) {
std::cout << " " << name << ": " << value << "\n";
}
std::cout << "\n";
}
// Assertions
if (!assertions.empty()) {
std::cout << "Assertions:\n";
bool allPassed = true;
for (const auto& [name, passed] : assertions) {
std::cout << " " << (passed ? "✓" : "✗") << " " << name << "\n";
if (!passed) allPassed = false;
}
std::cout << "\n";
if (allPassed) {
std::cout << "Result: ✅ PASSED\n";
} else {
std::cout << "Result: ❌ FAILED\n";
}
}
std::cout << "════════════════════════════════════════════════════════════════\n";
}
int TestReporter::getExitCode() const {
for (const auto& [name, passed] : assertions) {
if (!passed) return 1; // FAIL
}
return 0; // PASS
}
} // namespace grove

24
tests/helpers/TestReporter.h Normal file
View File

@ -0,0 +1,24 @@
#pragma once
#include <string>
#include <map>
#include <vector>
namespace grove {
class TestReporter {
public:
explicit TestReporter(const std::string& scenarioName);
void addMetric(const std::string& name, float value);
void addAssertion(const std::string& name, bool passed);
void printFinalReport() const;
int getExitCode() const; // 0 = pass, 1 = fail
private:
std::string scenarioName;
std::map<std::string, float> metrics;
std::vector<std::pair<std::string, bool>> assertions;
};
} // namespace grove

8
tests/hotreload/test_engine_hotreload.cpp
View File

@ -104,6 +104,9 @@ int main(int argc, char** argv) {
std::cout << "✅ Hot-reload completed in " << reloadTime << "ms" << std::endl; std::cout << "✅ Hot-reload completed in " << reloadTime << "ms" << std::endl;
std::cout << "📊 State should be preserved - check counter continues!\n" << std::endl; std::cout << "📊 State should be preserved - check counter continues!\n" << std::endl;
// Reset watcher to avoid re-detecting the same change
watcher.reset(modulePath);
} catch (const std::exception& e) { } catch (const std::exception& e) {
std::cerr << "❌ Hot-reload failed: " << e.what() << std::endl; std::cerr << "❌ Hot-reload failed: " << e.what() << std::endl;
} }
@ -126,6 +129,11 @@ int main(int argc, char** argv) {
<< " | Runtime: " << static_cast<int>(totalElapsed) << "s" << " | Runtime: " << static_cast<int>(totalElapsed) << "s"
<< " | FPS: " << static_cast<int>(actualFPS) << std::endl; << " | FPS: " << static_cast<int>(actualFPS) << std::endl;
// Dump module state every 2 seconds
std::cout << "\n📊 Dumping module state:\n" << std::endl;
debugEngine->dumpModuleState(moduleName);
std::cout << std::endl;
lastStatusTime = currentTime; lastStatusTime = currentTime;
} }

170
tests/hotreload/test_hotreload_live.sh Normal file
View File

@ -0,0 +1,170 @@
#!/bin/bash
# Test script for live hot-reload during engine execution
# This script validates that hot-reload works while the engine is running
set -e
echo "========================================"
echo "🔥 LIVE HOT-RELOAD TEST"
echo "========================================"
echo "This test will:"
echo " 1. Start the engine"
echo " 2. Modify TestModule.cpp while running"
echo " 3. Recompile the module"
echo " 4. Verify hot-reload happens automatically"
echo "========================================"
echo ""
# Configuration
BUILD_DIR="../../build"
TEST_DIR="$BUILD_DIR/tests"
MODULE_SOURCE="../../tests/modules/TestModule.cpp"
ENGINE_EXEC="$TEST_DIR/test_engine_hotreload"
LOG_FILE="/tmp/grove_hotreload_test.log"
# Check that we're in the right directory
if [ ! -f "$ENGINE_EXEC" ]; then
echo "❌ Error: test_engine_hotreload not found at $ENGINE_EXEC"
echo "Please run this script from tests/hotreload/ directory"
exit 1
fi
# Clean previous log
rm -f "$LOG_FILE"
# Step 1: Start the engine in background
echo "🚀 Step 1/5: Starting engine in background..."
cd "$TEST_DIR"
./test_engine_hotreload > "$LOG_FILE" 2>&1 &
ENGINE_PID=$!
echo " Engine PID: $ENGINE_PID"
echo " Log file: $LOG_FILE"
# Give engine time to fully start
echo "⏳ Waiting 3 seconds for engine to start..."
sleep 3
# Check if engine is still running
if ! kill -0 $ENGINE_PID 2>/dev/null; then
echo "❌ Engine died during startup!"
cat "$LOG_FILE"
exit 1
fi
echo "✅ Engine is running"
echo ""
# Step 2: Check initial state
echo "📊 Step 2/5: Checking initial state..."
INITIAL_COUNT=$(grep -c "Version: v1.0" "$LOG_FILE" || echo "0")
echo " Initial frames processed with v1.0: $INITIAL_COUNT"
if [ "$INITIAL_COUNT" -lt 10 ]; then
echo "❌ Engine not processing frames properly (expected >= 10, got $INITIAL_COUNT)"
kill $ENGINE_PID 2>/dev/null || true
exit 1
fi
echo "✅ Engine processing frames normally"
echo ""
# Step 3: Modify TestModule.cpp
echo "🔧 Step 3/5: Modifying TestModule.cpp..."
TEMP_MODULE="/tmp/TestModule_backup.cpp"
cp "$MODULE_SOURCE" "$TEMP_MODULE"
# Change version from v1.0 to v2.0 HOT-RELOADED!
sed -i 's/v1\.0/v2.0 HOT-RELOADED!/g' "$MODULE_SOURCE"
echo "✅ TestModule.cpp modified (v1.0 → v2.0 HOT-RELOADED!)"
echo ""
# Step 4: Recompile module
echo "🔨 Step 4/5: Recompiling TestModule..."
cd "$BUILD_DIR"
make TestModule > /dev/null 2>&1
if [ $? -ne 0 ]; then
echo "❌ Compilation failed!"
# Restore original
cp "$TEMP_MODULE" "$MODULE_SOURCE"
kill $ENGINE_PID 2>/dev/null || true
exit 1
fi
echo "✅ TestModule recompiled successfully"
echo ""
# Step 5: Wait and verify hot-reload happened
echo "⏳ Step 5/5: Waiting for hot-reload detection..."
sleep 2
# Check if hot-reload was triggered
if grep -q "Hot-reload completed" "$LOG_FILE"; then
echo "✅ Hot-reload was triggered!"
# Check if new version is running
if grep -q "v2.0 HOT-RELOADED!" "$LOG_FILE"; then
echo "✅ New version (v2.0) is running!"
# Count frames with new version
V2_COUNT=$(grep -c "v2.0 HOT-RELOADED!" "$LOG_FILE" || echo "0")
echo " Frames processed with v2.0: $V2_COUNT"
if [ "$V2_COUNT" -ge 5 ]; then
echo ""
echo "========================================"
echo "🎉 HOT-RELOAD TEST PASSED!"
echo "========================================"
echo "✅ Engine ran with v1.0"
echo "✅ Module was modified and recompiled"
echo "✅ Hot-reload was detected and executed"
echo "✅ Engine continued running with v2.0"
echo "========================================"
# Show reload timing
RELOAD_TIME=$(grep "Hot-reload completed in" "$LOG_FILE" | tail -1 | grep -oP '\d+\.\d+(?=ms)')
if [ -n "$RELOAD_TIME" ]; then
echo "⚡ Hot-reload time: ${RELOAD_TIME}ms"
fi
SUCCESS=true
else
echo "❌ Not enough frames with v2.0 (expected >= 5, got $V2_COUNT)"
SUCCESS=false
fi
else
echo "❌ New version not detected in output"
SUCCESS=false
fi
else
echo "❌ Hot-reload was not triggered"
echo ""
echo "Last 30 lines of log:"
tail -30 "$LOG_FILE"
SUCCESS=false
fi
# Cleanup
echo ""
echo "🧹 Cleaning up..."
kill $ENGINE_PID 2>/dev/null || true
wait $ENGINE_PID 2>/dev/null || true
# Restore original module
cp "$TEMP_MODULE" "$MODULE_SOURCE"
rm -f "$TEMP_MODULE"
# Recompile original version
echo "🔄 Restoring original TestModule..."
cd "$BUILD_DIR"
make TestModule > /dev/null 2>&1
if [ "$SUCCESS" = true ]; then
echo "✅ Test completed successfully"
exit 0
else
echo "❌ Test failed"
exit 1
fi

264
tests/integration/test_01_production_hotreload.cpp Normal file
View File

@ -0,0 +1,264 @@
#include "grove/ModuleLoader.h"
#include "grove/SequentialModuleSystem.h"
#include "grove/JsonDataNode.h"
#include "../helpers/TestMetrics.h"
#include "../helpers/TestAssertions.h"
#include "../helpers/TestReporter.h"
#include "../helpers/SystemUtils.h"
#include <iostream>
#include <chrono>
#include <thread>
#include <fstream>
#include <regex>
using namespace grove;
int main() {
TestReporter reporter("Production Hot-Reload");
TestMetrics metrics;
std::cout << "================================================================================\n";
std::cout << "TEST: Production Hot-Reload\n";
std::cout << "================================================================================\n\n";
// === SETUP ===
std::cout << "Setup: Loading TankModule...\n";
ModuleLoader loader;
auto moduleSystem = std::make_unique<SequentialModuleSystem>();
// Charger module
std::string modulePath = "build/tests/libTankModule.so";
auto module = loader.load(modulePath, "TankModule", false);
// Config
nlohmann::json configJson;
configJson["tankCount"] = 50;
configJson["version"] = "v1.0";
auto config = std::make_unique<JsonDataNode>("config", configJson);
// Initialiser (setConfiguration)
module->setConfiguration(*config, nullptr, nullptr);
// Enregistrer dans system
moduleSystem->registerModule("TankModule", std::move(module));
std::cout << " ✓ Module loaded and initialized\n\n";
// === PHASE 1: Pre-Reload (15s = 900 frames) ===
std::cout << "Phase 1: Running 15s before reload...\n";
// Créer input avec deltaTime
nlohmann::json inputJson;
inputJson["deltaTime"] = 1.0 / 60.0;
auto inputNode = std::make_unique<JsonDataNode>("input", inputJson);
for (int frame = 0; frame < 900; frame++) {
auto frameStart = std::chrono::high_resolution_clock::now();
moduleSystem->processModules(1.0f / 60.0f);
auto frameEnd = std::chrono::high_resolution_clock::now();
float frameTime = std::chrono::duration<float, std::milli>(frameEnd - frameStart).count();
metrics.recordFPS(1000.0f / frameTime);
if (frame % 60 == 0) {
metrics.recordMemoryUsage(getCurrentMemoryUsage());
}
if (frame % 300 == 0) {
std::cout << " Frame " << frame << "/900\n";
}
}
// Snapshot state AVANT reload
auto tankModule = moduleSystem->extractModule();
auto preReloadState = tankModule->getState();
// Cast to JsonDataNode to access JSON
auto* jsonNodeBefore = dynamic_cast<JsonDataNode*>(preReloadState.get());
if (!jsonNodeBefore) {
std::cerr << "❌ Failed to cast state to JsonDataNode\n";
return 1;
}
const auto& stateJsonBefore = jsonNodeBefore->getJsonData();
int tankCountBefore = stateJsonBefore["tanks"].size();
std::string versionBefore = stateJsonBefore.value("version", "unknown");
int frameCountBefore = stateJsonBefore.value("frameCount", 0);
std::cout << "\nState snapshot BEFORE reload:\n";
std::cout << " Version: " << versionBefore << "\n";
std::cout << " Tank count: " << tankCountBefore << "\n";
std::cout << " Frame: " << frameCountBefore << "\n\n";
ASSERT_EQ(tankCountBefore, 50, "Should have 50 tanks before reload");
// Ré-enregistrer le module temporairement
moduleSystem->registerModule("TankModule", std::move(tankModule));
// === HOT-RELOAD ===
std::cout << "Triggering hot-reload...\n";
// Modifier version dans source (HEADER)
std::cout << " 1. Modifying source code (v1.0 -> v2.0 HOT-RELOADED)...\n";
std::ifstream input("tests/modules/TankModule.h");
std::string content((std::istreambuf_iterator<char>(input)), std::istreambuf_iterator<char>());
input.close();
size_t pos = content.find("std::string moduleVersion = \"v1.0\";");
if (pos != std::string::npos) {
content.replace(pos, 39, "std::string moduleVersion = \"v2.0 HOT-RELOADED\";");
}
std::ofstream output("tests/modules/TankModule.h");
output << content;
output.close();
// Recompiler
std::cout << " 2. Recompiling module...\n";
int buildResult = system("cmake --build build --target TankModule 2>&1 > /dev/null");
if (buildResult != 0) {
std::cerr << "❌ Compilation failed!\n";
return 1;
}
std::cout << " ✓ Compilation succeeded\n";
// Wait for file to be ready (simulate file stability check)
std::this_thread::sleep_for(std::chrono::milliseconds(500));
// Reload
std::cout << " 3. Reloading module...\n";
auto reloadStart = std::chrono::high_resolution_clock::now();
// Extract module from system
tankModule = moduleSystem->extractModule();
// Use ModuleLoader::reload()
auto newModule = loader.reload(std::move(tankModule));
// Re-register
moduleSystem->registerModule("TankModule", std::move(newModule));
auto reloadEnd = std::chrono::high_resolution_clock::now();
float reloadTime = std::chrono::duration<float, std::milli>(reloadEnd - reloadStart).count();
metrics.recordReloadTime(reloadTime);
reporter.addMetric("reload_time_ms", reloadTime);
std::cout << " ✓ Reload completed in " << reloadTime << "ms\n\n";
// === VÉRIFICATIONS POST-RELOAD ===
std::cout << "Verifying state preservation...\n";
tankModule = moduleSystem->extractModule();
auto postReloadState = tankModule->getState();
auto* jsonNodeAfter = dynamic_cast<JsonDataNode*>(postReloadState.get());
if (!jsonNodeAfter) {
std::cerr << "❌ Failed to cast post-reload state to JsonDataNode\n";
return 1;
}
const auto& stateJsonAfter = jsonNodeAfter->getJsonData();
int tankCountAfter = stateJsonAfter["tanks"].size();
std::string versionAfter = stateJsonAfter.value("version", "unknown");
int frameCountAfter = stateJsonAfter.value("frameCount", 0);
std::cout << "\nState snapshot AFTER reload:\n";
std::cout << " Version: " << versionAfter << "\n";
std::cout << " Tank count: " << tankCountAfter << "\n";
std::cout << " Frame: " << frameCountAfter << "\n\n";
// Vérification 1: Nombre de tanks
ASSERT_EQ(tankCountAfter, 50, "Should still have 50 tanks after reload");
reporter.addAssertion("tank_count_preserved", tankCountAfter == 50);
// Vérification 2: Version mise à jour
bool versionUpdated = versionAfter.find("v2.0") != std::string::npos;
ASSERT_TRUE(versionUpdated, "Version should be updated to v2.0");
reporter.addAssertion("version_updated", versionUpdated);
// Vérification 3: Frame count préservé
ASSERT_EQ(frameCountAfter, frameCountBefore, "Frame count should be preserved");
reporter.addAssertion("framecount_preserved", frameCountAfter == frameCountBefore);
std::cout << " ✓ State preserved correctly\n";
// Ré-enregistrer module
moduleSystem->registerModule("TankModule", std::move(tankModule));
// === PHASE 2: Post-Reload (15s = 900 frames) ===
std::cout << "\nPhase 2: Running 15s after reload...\n";
for (int frame = 0; frame < 900; frame++) {
auto frameStart = std::chrono::high_resolution_clock::now();
moduleSystem->processModules(1.0f / 60.0f);
auto frameEnd = std::chrono::high_resolution_clock::now();
float frameTime = std::chrono::duration<float, std::milli>(frameEnd - frameStart).count();
metrics.recordFPS(1000.0f / frameTime);
if (frame % 60 == 0) {
metrics.recordMemoryUsage(getCurrentMemoryUsage());
}
if (frame % 300 == 0) {
std::cout << " Frame " << frame << "/900\n";
}
}
// === VÉRIFICATIONS FINALES ===
std::cout << "\nFinal verifications...\n";
// Memory growth
size_t memGrowth = metrics.getMemoryGrowth();
float memGrowthMB = memGrowth / (1024.0f * 1024.0f);
ASSERT_LT(memGrowthMB, 5.0f, "Memory growth should be < 5MB");
reporter.addMetric("memory_growth_mb", memGrowthMB);
// FPS
float minFPS = metrics.getFPSMin();
ASSERT_GT(minFPS, 30.0f, "Min FPS should be > 30");
reporter.addMetric("fps_min", minFPS);
reporter.addMetric("fps_avg", metrics.getFPSAvg());
reporter.addMetric("fps_max", metrics.getFPSMax());
// Reload time
ASSERT_LT(reloadTime, 1000.0f, "Reload time should be < 1000ms");
// No crashes
reporter.addAssertion("no_crashes", true);
// === CLEANUP ===
std::cout << "\nCleaning up...\n";
// Restaurer version originale (HEADER)
std::ifstream inputRestore("tests/modules/TankModule.h");
std::string contentRestore((std::istreambuf_iterator<char>(inputRestore)), std::istreambuf_iterator<char>());
inputRestore.close();
pos = contentRestore.find("std::string moduleVersion = \"v2.0 HOT-RELOADED\";");
if (pos != std::string::npos) {
contentRestore.replace(pos, 50, "std::string moduleVersion = \"v1.0\";");
}
std::ofstream outputRestore("tests/modules/TankModule.h");
outputRestore << contentRestore;
outputRestore.close();
system("cmake --build build --target TankModule 2>&1 > /dev/null");
// === RAPPORTS ===
std::cout << "\n";
metrics.printReport();
reporter.printFinalReport();
return reporter.getExitCode();
}

255
tests/integration/test_02_chaos_monkey.cpp Normal file
View File

@ -0,0 +1,255 @@
// PREUVE : Décommenter cette ligne pour désactiver la recovery et voir le test ÉCHOUER
// #define DISABLE_RECOVERY_FOR_TEST
#include "grove/ModuleLoader.h"
#include "grove/SequentialModuleSystem.h"
#include "grove/JsonDataNode.h"
#include "../helpers/TestMetrics.h"
#include "../helpers/TestAssertions.h"
#include "../helpers/TestReporter.h"
#include "../helpers/SystemUtils.h"
#include <iostream>
#include <chrono>
#include <thread>
#include <csignal>
#include <atomic>
using namespace grove;
// Global for crash detection
static std::atomic<bool> engineCrashed{false};
void signalHandler(int signal) {
if (signal == SIGSEGV || signal == SIGABRT) {
engineCrashed.store(true);
std::cerr << "❌ FATAL: Signal " << signal << " received (SIGSEGV or SIGABRT)\n";
std::cerr << "Engine has crashed unrecoverably.\n";
std::exit(1);
}
}
int main() {
TestReporter reporter("Chaos Monkey");
TestMetrics metrics;
std::cout << "================================================================================\n";
std::cout << "TEST: Chaos Monkey\n";
std::cout << "================================================================================\n\n";
// Setup signal handlers
std::signal(SIGSEGV, signalHandler);
std::signal(SIGABRT, signalHandler);
// === SETUP ===
std::cout << "Setup: Loading ChaosModule...\n";
ModuleLoader loader;
auto moduleSystem = std::make_unique<SequentialModuleSystem>();
// Load module
std::string modulePath = "build/tests/libChaosModule.so";
auto module = loader.load(modulePath, "ChaosModule", false);
// Configure module avec seed ALÉATOIRE basé sur le temps
// Chaque run sera différent - VRAI chaos
unsigned int randomSeed = static_cast<unsigned int>(std::chrono::system_clock::now().time_since_epoch().count());
nlohmann::json configJson;
configJson["seed"] = randomSeed;
configJson["hotReloadProbability"] = 0.30; // Non utilisé maintenant
configJson["crashProbability"] = 0.05; // 5% par frame = crash fréquent
configJson["corruptionProbability"] = 0.10; // Non utilisé
configJson["invalidConfigProbability"] = 0.05; // Non utilisé
auto config = std::make_unique<JsonDataNode>("config", configJson);
std::cout << " Random seed: " << randomSeed << " (time-based, unpredictable)\n";
module->setConfiguration(*config, nullptr, nullptr);
// Register in module system
moduleSystem->registerModule("ChaosModule", std::move(module));
std::cout << " ✓ ChaosModule loaded and configured\n\n";
// === CHAOS LOOP (30 seconds = 1800 frames @ 60 FPS) ===
// NOTE: Reduced from 5 minutes for faster testing
std::cout << "Starting Chaos Monkey (30 seconds simulation)...\n";
std::cout << "REAL CHAOS MODE:\n";
std::cout << " - 5% crash probability PER FRAME (not per second)\n";
std::cout << " - Expected crashes: ~90 crashes (5% of 1800 frames)\n";
std::cout << " - Random seed (time-based): unpredictable pattern\n";
std::cout << " - Multiple crash types: runtime_error, logic_error, out_of_range, domain_error, state corruption\n";
std::cout << " - Corrupted state validation: module must reject corrupted state\n\n";
const int totalFrames = 1800; // 30 * 60
int crashesDetected = 0;
int reloadsTriggered = 0;
int recoverySuccesses = 0;
bool hadDeadlock = false;
auto testStart = std::chrono::high_resolution_clock::now();
for (int frame = 0; frame < totalFrames; frame++) {
auto frameStart = std::chrono::high_resolution_clock::now();
bool didRecoveryThisFrame = false;
try {
// Process module (1/60th of a second)
moduleSystem->processModules(1.0f / 60.0f);
} catch (const std::exception& e) {
// CRASH DETECTED - Attempt recovery
crashesDetected++;
std::cout << " [Frame " << frame << "] ⚠️ Crash detected: " << e.what() << "\n";
// PREUVE QUE LE TEST PEUT ÉCHOUER : désactiver la recovery
#ifdef DISABLE_RECOVERY_FOR_TEST
std::cout << " [Frame " << frame << "] ❌ RECOVERY DISABLED - Test will fail\n";
reporter.addAssertion("recovery_disabled", false);
break; // Le test DOIT échouer
#endif
// Recovery attempt
try {
std::cout << " [Frame " << frame << "] 🔄 Attempting recovery...\n";
auto recoveryStart = std::chrono::high_resolution_clock::now();
// Extract module from system
auto crashedModule = moduleSystem->extractModule();
// Reload module
auto reloadedModule = loader.reload(std::move(crashedModule));
// Re-register
moduleSystem->registerModule("ChaosModule", std::move(reloadedModule));
auto recoveryEnd = std::chrono::high_resolution_clock::now();
float recoveryTime = std::chrono::duration<float, std::milli>(recoveryEnd - recoveryStart).count();
metrics.recordReloadTime(recoveryTime);
recoverySuccesses++;
didRecoveryThisFrame = true;
std::cout << " [Frame " << frame << "] ✅ Recovery successful (" << recoveryTime << "ms)\n";
} catch (const std::exception& recoveryError) {
std::cout << " [Frame " << frame << "] ❌ Recovery FAILED: " << recoveryError.what() << "\n";
reporter.addAssertion("recovery_failed", false);
break; // Stop test - recovery failed
}
}
// Metrics
auto frameEnd = std::chrono::high_resolution_clock::now();
float frameTime = std::chrono::duration<float, std::milli>(frameEnd - frameStart).count();
// Only record FPS for normal frames (not recovery frames)
// Recovery frames are slow by design (100+ ms for hot-reload)
if (!didRecoveryThisFrame) {
metrics.recordFPS(1000.0f / frameTime);
}
if (frame % 60 == 0) {
metrics.recordMemoryUsage(getCurrentMemoryUsage());
}
// Deadlock detection (frame > 100ms)
// NOTE: Skip deadlock check if we just did a recovery (recovery takes >100ms by design)
if (frameTime > 100.0f && !didRecoveryThisFrame) {
std::cout << " [Frame " << frame << "] ⚠️ Potential deadlock (frame time: " << frameTime << "ms)\n";
hadDeadlock = true;
}
// Progress (every 600 frames = 10 seconds)
if (frame % 600 == 0 && frame > 0) {
float elapsedSec = frame / 60.0f;
float progress = (frame * 100.0f) / totalFrames;
std::cout << "Progress: " << elapsedSec << "/30.0 seconds (" << (int)progress << "%)\n";
// Show current metrics
std::cout << " FPS: min=" << metrics.getFPSMin() << ", avg=" << metrics.getFPSAvg() << ", max=" << metrics.getFPSMax() << "\n";
std::cout << " Memory: " << (getCurrentMemoryUsage() / (1024.0f * 1024.0f)) << " MB\n";
}
// Check if engine crashed externally
if (engineCrashed.load()) {
std::cout << " [Frame " << frame << "] ❌ Engine crashed externally (signal received)\n";
reporter.addAssertion("engine_crashed_externally", false);
break;
}
}
auto testEnd = std::chrono::high_resolution_clock::now();
float totalDuration = std::chrono::duration<float>(testEnd - testStart).count();
std::cout << "\nTest completed!\n\n";
// === FINAL VERIFICATIONS ===
std::cout << "Final verifications...\n";
// Engine still alive
bool engineAlive = !engineCrashed.load();
ASSERT_TRUE(engineAlive, "Engine should still be alive");
reporter.addAssertion("engine_alive", engineAlive);
// No deadlocks
ASSERT_FALSE(hadDeadlock, "Should not have deadlocks");
reporter.addAssertion("no_deadlocks", !hadDeadlock);
// Memory growth < 10MB
size_t memGrowth = metrics.getMemoryGrowth();
float memGrowthMB = memGrowth / (1024.0f * 1024.0f);
ASSERT_LT(memGrowthMB, 10.0f, "Memory growth should be < 10MB");
reporter.addMetric("memory_growth_mb", memGrowthMB);
// Test runs as fast as possible (not real-time)
// Just check it completed within reasonable bounds (< 60 seconds wall time)
ASSERT_LT(totalDuration, 60.0f, "Total duration should be < 60 seconds");
reporter.addMetric("total_duration_sec", totalDuration);
// FPS metrics
float minFPS = metrics.getFPSMin();
float avgFPS = metrics.getFPSAvg();
float maxFPS = metrics.getFPSMax();
// Min FPS should be reasonable (> 10 even with crashes)
ASSERT_GT(minFPS, 10.0f, "Min FPS should be > 10");
reporter.addMetric("fps_min", minFPS);
reporter.addMetric("fps_avg", avgFPS);
reporter.addMetric("fps_max", maxFPS);
// Recovery rate > 95%
float recoveryRate = (crashesDetected > 0) ? (recoverySuccesses * 100.0f / crashesDetected) : 100.0f;
ASSERT_GT(recoveryRate, 95.0f, "Recovery rate should be > 95%");
reporter.addMetric("recovery_rate_percent", recoveryRate);
// === STATISTICS ===
std::cout << "\n";
std::cout << "================================================================================\n";
std::cout << "CHAOS MONKEY STATISTICS\n";
std::cout << "================================================================================\n";
std::cout << " Total frames: " << totalFrames << "\n";
std::cout << " Duration: " << totalDuration << "s (wall time, not simulation time)\n";
std::cout << " Crashes detected: " << crashesDetected << "\n";
std::cout << " Recovery successes: " << recoverySuccesses << "\n";
std::cout << " Recovery rate: " << recoveryRate << "%\n";
std::cout << " Memory growth: " << memGrowthMB << " MB (max: 10MB)\n";
std::cout << " Had deadlocks: " << (hadDeadlock ? "YES ❌" : "NO ✅") << "\n";
std::cout << " FPS min/avg/max: " << minFPS << " / " << avgFPS << " / " << maxFPS << "\n";
std::cout << "================================================================================\n\n";
std::cout << "Note: ChaosModule generates random crashes internally.\n";
std::cout << "The test should recover from ALL crashes automatically via hot-reload.\n\n";
// === CLEANUP ===
std::cout << "Cleaning up...\n";
moduleSystem.reset();
std::cout << " ✓ Module system shutdown complete\n\n";
// === REPORTS ===
metrics.printReport();
reporter.printFinalReport();
return reporter.getExitCode();
}

247
tests/integration/test_03_stress_test.cpp Normal file
View File

@ -0,0 +1,247 @@
/**
* @file test_03_stress_test.cpp
* @brief Scenario 3: Stress Test - Long-duration stability validation
*
* OBJECTIVE:
* Validate hot-reload system stability over extended duration with repeated reloads.
*
* TEST PARAMETERS:
* - Duration: 10 minutes (36000 frames @ 60 FPS)
* - Reload frequency: Every 5 seconds (300 frames)
* - Total reloads: 120
* - No random crashes - focus on hot-reload stability
*
* SUCCESS CRITERIA:
* ✅ All 120 reloads succeed
* ✅ Memory growth < 50MB over 10 minutes
* ✅ Average reload time < 500ms
* ✅ FPS remains stable (no degradation)
* ✅ No file descriptor leaks
* ✅ State preserved across all reloads
*
* WHAT THIS VALIDATES:
* - No memory leaks in hot-reload system
* - No file descriptor leaks (dlopen/dlclose)
* - Reload performance doesn't degrade over time
* - State preservation is reliable at scale
* - System remains stable under repeated reload stress
*/
#include "grove/ModuleLoader.h"
#include "grove/SequentialModuleSystem.h"
#include "grove/JsonDataNode.h"
#include "../helpers/TestMetrics.h"
#include "../helpers/TestAssertions.h"
#include "../helpers/TestReporter.h"
#include "../helpers/SystemUtils.h"
#include <iostream>
#include <chrono>
#include <thread>
using namespace grove;
// Test configuration
constexpr int TARGET_FPS = 60;
constexpr float FRAME_TIME = 1.0f / TARGET_FPS;
constexpr int RELOAD_INTERVAL = 300; // Reload every 5 seconds (300 frames)
constexpr int EXPECTED_RELOADS = 120; // 120 reloads
constexpr int TOTAL_FRAMES = EXPECTED_RELOADS * RELOAD_INTERVAL; // 36000 frames = 10 minutes @ 60 FPS
// Memory threshold
constexpr size_t MAX_MEMORY_GROWTH_MB = 50;
// Paths
const std::string MODULE_PATH = "build/tests/libStressModule.so";
int main() {
TestReporter reporter("Stress Test - 10 Minute Stability");
TestMetrics metrics;
std::cout << "═══════════════════════════════════════════════════════════════\n";
std::cout << " SCENARIO 3: STRESS TEST - LONG DURATION STABILITY\n";
std::cout << "═══════════════════════════════════════════════════════════════\n";
std::cout << "Duration: 10 minutes (" << TOTAL_FRAMES << " frames @ " << TARGET_FPS << " FPS)\n";
std::cout << "Reload interval: Every " << RELOAD_INTERVAL << " frames (5 seconds)\n";
std::cout << "Expected reloads: " << EXPECTED_RELOADS << "\n";
std::cout << "Memory threshold: < " << MAX_MEMORY_GROWTH_MB << " MB growth\n";
std::cout << "═══════════════════════════════════════════════════════════════\n\n";
size_t initialMemory = grove::getCurrentMemoryUsage() / (1024 * 1024);
size_t peakMemory = initialMemory;
int successfulReloads = 0;
int failedReloads = 0;
try {
// === SETUP ===
std::cout << "Setup: Loading StressModule...\n";
ModuleLoader loader;
auto moduleSystem = std::make_unique<SequentialModuleSystem>();
// Load module
auto module = loader.load(MODULE_PATH, "StressModule", false);
// Configure module with empty config
nlohmann::json configJson;
auto config = std::make_unique<JsonDataNode>("config", configJson);
module->setConfiguration(*config, nullptr, nullptr);
// Register in module system
moduleSystem->registerModule("StressModule", std::move(module));
std::cout << " ✓ StressModule loaded and configured\n\n";
std::cout << "🚀 Starting 10-minute stress test...\n\n";
auto startTime = std::chrono::high_resolution_clock::now();
// Main simulation loop
for (int frame = 1; frame <= TOTAL_FRAMES; ++frame) {
auto frameStart = std::chrono::high_resolution_clock::now();
// Process modules
try {
moduleSystem->processModules(FRAME_TIME);
} catch (const std::exception& e) {
std::cerr << " [Frame " << frame << "] ❌ Unexpected error during process: " << e.what() << "\n";
reporter.addAssertion("process_error", false);
break;
}
auto frameEnd = std::chrono::high_resolution_clock::now();
auto frameDuration = std::chrono::duration<float, std::milli>(frameEnd - frameStart).count();
float fps = frameDuration > 0.0f ? 1000.0f / frameDuration : 0.0f;
metrics.recordFPS(fps);
// Hot-reload every RELOAD_INTERVAL frames
if (frame % RELOAD_INTERVAL == 0) {
int reloadNumber = frame / RELOAD_INTERVAL;
std::cout << " [Frame " << frame << "/" << TOTAL_FRAMES << "] 🔄 Triggering hot-reload #" << reloadNumber << "...\n";
auto reloadStart = std::chrono::high_resolution_clock::now();
try {
// Extract module from system
auto extractedModule = moduleSystem->extractModule();
if (!extractedModule) {
std::cerr << " ❌ Failed to extract StressModule\n";
failedReloads++;
continue;
}
// Perform hot-reload
auto reloadedModule = loader.reload(std::move(extractedModule));
// Re-register reloaded module
moduleSystem->registerModule("StressModule", std::move(reloadedModule));
auto reloadEnd = std::chrono::high_resolution_clock::now();
auto reloadDuration = std::chrono::duration_cast<std::chrono::milliseconds>(
reloadEnd - reloadStart).count();
metrics.recordReloadTime(static_cast<float>(reloadDuration));
successfulReloads++;
std::cout << " ✅ Hot-reload #" << reloadNumber << " succeeded in " << reloadDuration << "ms\n";
} catch (const std::exception& e) {
std::cerr << " ❌ Exception during hot-reload: " << e.what() << "\n";
failedReloads++;
}
}
// Memory monitoring every 60 seconds (3600 frames)
if (frame % 3600 == 0 && frame > 0) {
size_t currentMemory = grove::getCurrentMemoryUsage() / (1024 * 1024);
size_t memoryGrowth = currentMemory - initialMemory;
peakMemory = std::max(peakMemory, currentMemory);
int minutesElapsed = frame / 3600;
std::cout << "\n📊 Checkpoint at " << minutesElapsed << " minute(s):\n";
std::cout << " Current memory: " << currentMemory << " MB\n";
std::cout << " Growth: " << memoryGrowth << " MB\n";
std::cout << " Peak: " << peakMemory << " MB\n";
std::cout << " Avg FPS: " << metrics.getFPSAvg() << "\n";
std::cout << " Reloads: " << successfulReloads << "/" << EXPECTED_RELOADS << "\n";
std::cout << " Avg reload time: " << metrics.getReloadTimeAvg() << "ms\n\n";
}
// Progress reporting every minute (without memory details)
if (frame % 3600 == 0 && frame > 0) {
int minutesElapsed = frame / 3600;
int minutesRemaining = (TOTAL_FRAMES - frame) / 3600;
std::cout << "⏱️ Progress: " << minutesElapsed << " minutes elapsed, " << minutesRemaining << " minutes remaining\n";
}
}
auto endTime = std::chrono::high_resolution_clock::now();
auto totalDuration = std::chrono::duration_cast<std::chrono::seconds>(
endTime - startTime).count();
// Final metrics
size_t finalMemory = grove::getCurrentMemoryUsage() / (1024 * 1024);
size_t totalMemoryGrowth = finalMemory - initialMemory;
std::cout << "\n═══════════════════════════════════════════════════════════════\n";
std::cout << " STRESS TEST COMPLETED\n";
std::cout << "═══════════════════════════════════════════════════════════════\n";
std::cout << "Total frames: " << TOTAL_FRAMES << "\n";
std::cout << "Real time: " << totalDuration << "s\n";
std::cout << "Simulated time: " << (TOTAL_FRAMES / TARGET_FPS) << "s (10 minutes)\n";
std::cout << "Successful reloads: " << successfulReloads << "/" << EXPECTED_RELOADS << "\n";
std::cout << "Failed reloads: " << failedReloads << "\n";
std::cout << "\n📊 PERFORMANCE METRICS:\n";
std::cout << "Average FPS: " << metrics.getFPSAvg() << "\n";
std::cout << "Min FPS: " << metrics.getFPSMin() << "\n";
std::cout << "Max FPS: " << metrics.getFPSMax() << "\n";
std::cout << "\n🔥 HOT-RELOAD METRICS:\n";
std::cout << "Avg reload time: " << metrics.getReloadTimeAvg() << "ms\n";
std::cout << "Min reload time: " << metrics.getReloadTimeMin() << "ms\n";
std::cout << "Max reload time: " << metrics.getReloadTimeMax() << "ms\n";
std::cout << "\n💾 MEMORY METRICS:\n";
std::cout << "Initial memory: " << initialMemory << " MB\n";
std::cout << "Final memory: " << finalMemory << " MB\n";
std::cout << "Peak memory: " << peakMemory << " MB\n";
std::cout << "Total growth: " << totalMemoryGrowth << " MB\n";
std::cout << "═══════════════════════════════════════════════════════════════\n\n";
// Validate results
bool allReloadsSucceeded = (successfulReloads == EXPECTED_RELOADS && failedReloads == 0);
bool memoryWithinThreshold = (totalMemoryGrowth < MAX_MEMORY_GROWTH_MB);
bool avgReloadTimeAcceptable = (metrics.getReloadTimeAvg() < 500.0f);
bool fpsStable = (metrics.getFPSMin() > 30.0f); // Ensure FPS doesn't drop too much
reporter.addAssertion("all_reloads_succeeded", allReloadsSucceeded);
reporter.addAssertion("memory_within_threshold", memoryWithinThreshold);
reporter.addAssertion("avg_reload_time_acceptable", avgReloadTimeAcceptable);
reporter.addAssertion("fps_stable", fpsStable);
if (allReloadsSucceeded && memoryWithinThreshold &&
avgReloadTimeAcceptable && fpsStable) {
std::cout << "✅ STRESS TEST PASSED - System is stable over 10 minutes\n";
} else {
if (!allReloadsSucceeded) {
std::cerr << "❌ Reload success rate: " << successfulReloads << "/" << EXPECTED_RELOADS << "\n";
}
if (!memoryWithinThreshold) {
std::cerr << "❌ Memory growth: " << totalMemoryGrowth << " MB (threshold: " << MAX_MEMORY_GROWTH_MB << " MB)\n";
}
if (!avgReloadTimeAcceptable) {
std::cerr << "❌ Avg reload time: " << metrics.getReloadTimeAvg() << "ms (threshold: 500ms)\n";
}
if (!fpsStable) {
std::cerr << "❌ Min FPS: " << metrics.getFPSMin() << " (threshold: 30.0)\n";
}
}
} catch (const std::exception& e) {
std::cerr << "Test failed with exception: " << e.what() << "\n";
reporter.addAssertion("exception", false);
}
reporter.printFinalReport();
return reporter.getExitCode();
}

213
tests/modules/ChaosModule.cpp Normal file
View File

@ -0,0 +1,213 @@
#include "ChaosModule.h"
#include "grove/JsonDataNode.h"
#include <spdlog/spdlog.h>
#include <spdlog/sinks/stdout_color_sinks.h>
#include <stdexcept>
namespace grove {
void ChaosModule::setConfiguration(const IDataNode& configNode, IIO* io, ITaskScheduler* scheduler) {
// Logger
logger = spdlog::get("ChaosModule");
if (!logger) {
logger = spdlog::stdout_color_mt("ChaosModule");
}
logger->set_level(spdlog::level::debug);
// Clone config
const auto* jsonConfigNode = dynamic_cast<const JsonDataNode*>(&configNode);
if (jsonConfigNode) {
config = std::make_unique<JsonDataNode>("config", jsonConfigNode->getJsonData());
} else {
config = std::make_unique<JsonDataNode>("config");
}
// Lire config
int seed = configNode.getInt("seed", 42);
hotReloadProbability = static_cast<float>(configNode.getDouble("hotReloadProbability", 0.30));
crashProbability = static_cast<float>(configNode.getDouble("crashProbability", 0.10));
corruptionProbability = static_cast<float>(configNode.getDouble("corruptionProbability", 0.10));
invalidConfigProbability = static_cast<float>(configNode.getDouble("invalidConfigProbability", 0.05));
// Initialiser RNG
rng.seed(seed);
logger->info("Initializing ChaosModule");
logger->info(" Seed: {}", seed);
logger->info(" Hot-reload probability: {}", hotReloadProbability);
logger->info(" Crash probability: {}", crashProbability);
logger->info(" Corruption probability: {}", corruptionProbability);
logger->info(" Invalid config probability: {}", invalidConfigProbability);
frameCount = 0;
crashCount = 0;
corruptionCount = 0;
hotReloadCount = 0;
}
const IDataNode& ChaosModule::getConfiguration() {
return *config;
}
void ChaosModule::process(const IDataNode& input) {
isProcessing = true;
frameCount++;
// VRAI CHAOS: Tire aléatoirement À CHAQUE FRAME
// Pas de "toutes les 60 frames", on peut crasher N'IMPORTE QUAND
std::uniform_real_distribution<float> dist(0.0f, 1.0f);
float roll = dist(rng);
// Crash aléatoire (10% par frame = crash très fréquent)
if (roll < crashProbability) {
triggerChaosEvent();
}
// Si état corrompu, on DOIT crasher
if (isCorrupted) {
logger->error("❌ FATAL: State is corrupted! Module cannot continue.");
throw std::runtime_error("FATAL: State corrupted - module is in invalid state");
}
isProcessing = false;
}
void ChaosModule::triggerChaosEvent() {
crashCount++;
// Plusieurs TYPES de crashes différents pour tester la robustesse
std::uniform_int_distribution<int> crashTypeDist(0, 4);
int crashType = crashTypeDist(rng);
switch (crashType) {
case 0:
logger->warn("💥 Chaos: CRASH - runtime_error");
throw std::runtime_error("CRASH: Simulated runtime error at frame " + std::to_string(frameCount));
case 1:
logger->warn("💥 Chaos: CRASH - logic_error");
throw std::logic_error("CRASH: Logic error - invalid state transition at frame " + std::to_string(frameCount));
case 2:
logger->warn("💥 Chaos: CRASH - out_of_range");
throw std::out_of_range("CRASH: Out of range access at frame " + std::to_string(frameCount));
case 3:
logger->warn("💥 Chaos: CRASH - domain_error");
throw std::domain_error("CRASH: Domain error in computation at frame " + std::to_string(frameCount));
case 4:
// STATE CORRUPTION (plus vicieux - l'état devient invalide)
logger->warn("☠️ Chaos: STATE CORRUPTION - module will fail on next frame");
corruptionCount++;
isCorrupted = true;
// Pas de throw ici - on va crasher à la PROCHAINE frame
break;
}
}
std::unique_ptr<IDataNode> ChaosModule::getHealthStatus() {
nlohmann::json healthJson;
healthJson["status"] = isCorrupted ? "corrupted" : "healthy";
healthJson["frameCount"] = frameCount;
healthJson["crashCount"] = crashCount;
healthJson["corruptionCount"] = corruptionCount;
healthJson["hotReloadCount"] = hotReloadCount;
return std::make_unique<JsonDataNode>("health", healthJson);
}
void ChaosModule::shutdown() {
logger->info("Shutting down ChaosModule");
logger->info(" Total frames: {}", frameCount);
logger->info(" Crashes: {}", crashCount);
logger->info(" Corruptions: {}", corruptionCount);
logger->info(" Hot-reloads: {}", hotReloadCount);
}
std::string ChaosModule::getType() const {
return "chaos";
}
std::unique_ptr<IDataNode> ChaosModule::getState() {
nlohmann::json json;
json["frameCount"] = frameCount;
json["crashCount"] = crashCount;
json["corruptionCount"] = corruptionCount;
json["hotReloadCount"] = hotReloadCount;
json["isCorrupted"] = isCorrupted;
json["seed"] = 42; // Pour reproductibilité
return std::make_unique<JsonDataNode>("state", json);
}
void ChaosModule::setState(const IDataNode& state) {
const auto* jsonNode = dynamic_cast<const JsonDataNode*>(&state);
if (!jsonNode) {
if (logger) {
logger->error("setState: Invalid state (not JsonDataNode)");
}
return;
}
const auto& json = jsonNode->getJsonData();
// Ensure logger is initialized (needed after hot-reload)
if (!logger) {
logger = spdlog::get("ChaosModule");
if (!logger) {
logger = spdlog::stdout_color_mt("ChaosModule");
}
}
// Ensure config is initialized (needed after hot-reload)
if (!config) {
config = std::make_unique<JsonDataNode>("config");
}
// VALIDATION CRITIQUE: Refuser l'état corrompu
bool wasCorrupted = json.value("isCorrupted", false);
if (wasCorrupted) {
logger->error("🚫 REJECTED: Cannot restore corrupted state!");
logger->error(" The module was corrupted before hot-reload.");
logger->error(" Resetting to clean state instead.");
// Reset à un état propre au lieu de restaurer la corruption
frameCount = 0;
crashCount = 0;
corruptionCount = 0;
hotReloadCount = 0;
isCorrupted = false;
int seed = json.value("seed", 42);
rng.seed(seed);
logger->warn("⚠️ State reset due to corruption - module continues with fresh state");
return;
}
// Restaurer state sain
frameCount = json.value("frameCount", 0);
crashCount = json.value("crashCount", 0);
corruptionCount = json.value("corruptionCount", 0);
hotReloadCount = json.value("hotReloadCount", 0);
isCorrupted = false; // Toujours false après validation
int seed = json.value("seed", 42);
rng.seed(seed);
logger->info("State restored: frame {}, crashes {}, corruptions {}, hotReloads {}",
frameCount, crashCount, corruptionCount, hotReloadCount);
}
} // namespace grove
// Export symbols
extern "C" {
grove::IModule* createModule() {
return new grove::ChaosModule();
}
void destroyModule(grove::IModule* module) {
delete module;
}
}

51
tests/modules/ChaosModule.h Normal file
View File

@ -0,0 +1,51 @@
#pragma once
#include "grove/IModule.h"
#include "grove/IDataNode.h"
#include <random>
#include <memory>
#include <spdlog/spdlog.h>
namespace grove {
class ChaosModule : public IModule {
public:
// IModule interface
void process(const IDataNode& input) override;
void setConfiguration(const IDataNode& configNode, IIO* io, ITaskScheduler* scheduler) override;
const IDataNode& getConfiguration() override;
std::unique_ptr<IDataNode> getHealthStatus() override;
void shutdown() override;
std::unique_ptr<IDataNode> getState() override;
void setState(const IDataNode& state) override;
std::string getType() const override;
bool isIdle() const override { return !isProcessing; }
private:
std::mt19937 rng;
int frameCount = 0;
int crashCount = 0;
int corruptionCount = 0;
int hotReloadCount = 0;
bool isProcessing = false;
bool isCorrupted = false;
// Configuration du chaos
float hotReloadProbability = 0.30f;
float crashProbability = 0.10f;
float corruptionProbability = 0.10f;
float invalidConfigProbability = 0.05f;
std::shared_ptr<spdlog::logger> logger;
std::unique_ptr<IDataNode> config;
// Simulations de failures
void triggerChaosEvent();
};
} // namespace grove
// Export symbols
extern "C" {
grove::IModule* createModule();
void destroyModule(grove::IModule* module);
}

173
tests/modules/StressModule.cpp Normal file
View File

@ -0,0 +1,173 @@
#include "StressModule.h"
#include <grove/JsonDataNode.h>
#include <spdlog/spdlog.h>
#include <spdlog/sinks/stdout_color_sinks.h>
namespace grove {
void StressModule::setConfiguration(const IDataNode& configNode, IIO* io, ITaskScheduler* scheduler) {
// Initialize logger
logger_ = spdlog::get("StressModule");
if (!logger_) {
logger_ = spdlog::stdout_color_mt("StressModule");
}
logger_->set_level(spdlog::level::info);
// Clone config
const auto* jsonConfigNode = dynamic_cast<const JsonDataNode*>(&configNode);
if (jsonConfigNode) {
config_ = std::make_unique<JsonDataNode>("config", jsonConfigNode->getJsonData());
} else {
config_ = std::make_unique<JsonDataNode>("config");
}
logger_->info("Initializing StressModule");
initializeDummyData();
}
const IDataNode& StressModule::getConfiguration() {
return *config_;
}
void StressModule::process(const IDataNode& input) {
isProcessing_ = true;
frameCount_++;
// Lightweight processing - just validate data integrity periodically
if (frameCount_ % 1000 == 0) {
if (!validateDummyData()) {
logger_->error("Data validation failed at frame {}", frameCount_);
}
}
// Log progress every 60 seconds (3600 frames @ 60 FPS)
if (frameCount_ % 3600 == 0) {
logger_->info("Progress: {} frames, {} reloads", frameCount_, reloadCount_);
}
isProcessing_ = false;
}
std::unique_ptr<IDataNode> StressModule::getHealthStatus() {
nlohmann::json healthJson;
healthJson["status"] = validateDummyData() ? "healthy" : "corrupted";
healthJson["frameCount"] = frameCount_;
healthJson["reloadCount"] = reloadCount_;
return std::make_unique<JsonDataNode>("health", healthJson);
}
void StressModule::shutdown() {
logger_->info("Shutting down StressModule");
logger_->info(" Total frames: {}", frameCount_);
logger_->info(" Total reloads: {}", reloadCount_);
}
std::string StressModule::getType() const {
return "stress";
}
std::unique_ptr<IDataNode> StressModule::getState() {
nlohmann::json json;
json["frameCount"] = frameCount_;
json["reloadCount"] = reloadCount_;
// Save dummy data as array
nlohmann::json dataArray = nlohmann::json::array();
for (size_t i = 0; i < DUMMY_DATA_SIZE; ++i) {
dataArray.push_back(dummyData_[i]);
}
json["dummyData"] = dataArray;
logger_->debug("State saved: frame={}, reload={}", frameCount_, reloadCount_);
return std::make_unique<JsonDataNode>("state", json);
}
void StressModule::setState(const IDataNode& state) {
const auto* jsonNode = dynamic_cast<const JsonDataNode*>(&state);
if (!jsonNode) {
if (logger_) {
logger_->error("setState: Invalid state (not JsonDataNode)");
}
return;
}
const auto& json = jsonNode->getJsonData();
// Ensure logger is initialized (needed after hot-reload)
if (!logger_) {
logger_ = spdlog::get("StressModule");
if (!logger_) {
logger_ = spdlog::stdout_color_mt("StressModule");
}
}
// Ensure config is initialized (needed after hot-reload)
if (!config_) {
config_ = std::make_unique<JsonDataNode>("config");
}
// Defensive: Initialize dummy data first
initializeDummyData();
// Restore state
frameCount_ = json.value("frameCount", 0);
reloadCount_ = json.value("reloadCount", 0);
// Increment reload count
reloadCount_++;
// Restore dummy data
if (json.contains("dummyData") && json["dummyData"].is_array()) {
const auto& dataArray = json["dummyData"];
if (dataArray.size() == DUMMY_DATA_SIZE) {
for (size_t i = 0; i < DUMMY_DATA_SIZE; ++i) {
dummyData_[i] = dataArray[i].get<int>();
}
} else {
logger_->warn("Dummy data size mismatch: expected {}, got {}",
DUMMY_DATA_SIZE, dataArray.size());
}
}
// Validate restored data
if (!validateDummyData()) {
logger_->error("Data validation failed after setState");
initializeDummyData(); // Re-initialize if corrupt
}
logger_->info("State restored: frame={}, reload={}", frameCount_, reloadCount_);
}
void StressModule::initializeDummyData() {
// Initialize with predictable pattern for validation
for (size_t i = 0; i < DUMMY_DATA_SIZE; ++i) {
dummyData_[i] = static_cast<int>(i * 42);
}
}
bool StressModule::validateDummyData() const {
for (size_t i = 0; i < DUMMY_DATA_SIZE; ++i) {
if (dummyData_[i] != static_cast<int>(i * 42)) {
if (logger_) {
logger_->error("Data corruption detected at index {}: expected {}, got {}",
i, i * 42, dummyData_[i]);
}
return false;
}
}
return true;
}
} // namespace grove
// Factory functions
extern "C" {
grove::IModule* createModule() {
return new grove::StressModule();
}
void destroyModule(grove::IModule* module) {
delete module;
}
}

55
tests/modules/StressModule.h Normal file
View File

@ -0,0 +1,55 @@
#pragma once
#include <grove/IModule.h>
#include <grove/IDataNode.h>
#include <memory>
#include <spdlog/spdlog.h>
namespace grove {
/**
* @brief Module for stress testing hot-reload stability over long duration
*
* This module is intentionally simple to focus on hot-reload mechanics:
* - No random crashes (unlike ChaosModule)
* - Minimal state (frameCount, reloadCount)
* - Lightweight processing
* - Focus: memory stability, reload reliability, performance consistency
*/
class StressModule : public IModule {
public:
// IModule interface
void process(const IDataNode& input) override;
void setConfiguration(const IDataNode& configNode, IIO* io, ITaskScheduler* scheduler) override;
const IDataNode& getConfiguration() override;
std::unique_ptr<IDataNode> getHealthStatus() override;
void shutdown() override;
std::unique_ptr<IDataNode> getState() override;
void setState(const IDataNode& state) override;
std::string getType() const override;
bool isIdle() const override { return !isProcessing_; }
private:
uint64_t frameCount_ = 0;
uint64_t reloadCount_ = 0;
float totalTime_ = 0.0f;
bool isProcessing_ = false;
// Simple dummy data to have some state to preserve
static constexpr size_t DUMMY_DATA_SIZE = 100;
int dummyData_[DUMMY_DATA_SIZE];
std::shared_ptr<spdlog::logger> logger_;
std::unique_ptr<IDataNode> config_;
void initializeDummyData();
bool validateDummyData() const;
};
} // namespace grove
// Factory function
extern "C" {
grove::IModule* createModule();
void destroyModule(grove::IModule* module);
}

218
tests/modules/TankModule.cpp Normal file
View File

@ -0,0 +1,218 @@
#include "TankModule.h"
#include "grove/JsonDataNode.h"
#include <spdlog/spdlog.h>
#include <spdlog/sinks/stdout_color_sinks.h>
#include <cmath>
namespace grove {
void TankModule::setConfiguration(const IDataNode& configNode, IIO* io, ITaskScheduler* scheduler) {
// Logger
logger = spdlog::get("TankModule");
if (!logger) {
logger = spdlog::stdout_color_mt("TankModule");
}
logger->set_level(spdlog::level::debug);
// Clone config en JSON (pour pouvoir retourner dans getConfiguration)
const auto* jsonConfigNode = dynamic_cast<const JsonDataNode*>(&configNode);
if (jsonConfigNode) {
config = std::make_unique<JsonDataNode>("config", jsonConfigNode->getJsonData());
} else {
// Fallback: créer un config vide
config = std::make_unique<JsonDataNode>("config");
}
// Lire config
int tankCount = configNode.getInt("tankCount", 50);
moduleVersion = configNode.getString("version", moduleVersion);
logger->info("Initializing TankModule {}", moduleVersion);
logger->info(" Tank count: {}", tankCount);
// Spawner tanks
spawnTanks(tankCount);
frameCount = 0;
}
const IDataNode& TankModule::getConfiguration() {
return *config;
}
void TankModule::process(const IDataNode& input) {
// Extract deltaTime from input (assurez-vous que l'input contient "deltaTime")
float deltaTime = static_cast<float>(input.getDouble("deltaTime", 1.0 / 60.0));
frameCount++;
// Update tous les tanks
for (auto& tank : tanks) {
updateTank(tank, deltaTime);
}
// Log toutes les 60 frames (1 seconde)
if (frameCount % 60 == 0) {
logger->trace("Frame {}: {} tanks active", frameCount, tanks.size());
}
}
std::unique_ptr<IDataNode> TankModule::getHealthStatus() {
nlohmann::json healthJson;
healthJson["status"] = "healthy";
healthJson["tankCount"] = tanks.size();
healthJson["frameCount"] = frameCount;
auto health = std::make_unique<JsonDataNode>("health", healthJson);
return health;
}
void TankModule::shutdown() {
logger->info("Shutting down TankModule");
tanks.clear();
}
std::string TankModule::getType() const {
return "tank";
}
std::unique_ptr<IDataNode> TankModule::getState() {
nlohmann::json json;
json["version"] = moduleVersion;
json["frameCount"] = frameCount;
// Sérialiser tanks
nlohmann::json tanksJson = nlohmann::json::array();
for (const auto& tank : tanks) {
tanksJson.push_back({
{"id", tank.id},
{"x", tank.x},
{"y", tank.y},
{"vx", tank.vx},
{"vy", tank.vy},
{"cooldown", tank.cooldown},
{"targetX", tank.targetX},
{"targetY", tank.targetY}
});
}
json["tanks"] = tanksJson;
return std::make_unique<JsonDataNode>("state", json);
}
void TankModule::setState(const IDataNode& state) {
// Cast to JsonDataNode to access underlying JSON
const auto* jsonNode = dynamic_cast<const JsonDataNode*>(&state);
if (!jsonNode) {
logger->error("setState: Invalid state (not JsonDataNode)");
return;
}
const auto& json = jsonNode->getJsonData();
// Ensure logger is initialized (needed after hot-reload)
if (!logger) {
logger = spdlog::get("TankModule");
if (!logger) {
logger = spdlog::stdout_color_mt("TankModule");
}
}
// Ensure config is initialized (needed after hot-reload)
if (!config) {
config = std::make_unique<JsonDataNode>("config");
}
// Restaurer state
// NOTE: Ne pas restaurer moduleVersion depuis le state!
// La version vient du CODE, pas du state. C'est voulu pour le hot-reload.
// moduleVersion est déjà initialisé à "v1.0" ou "v2.0 HOT-RELOADED" par le code
frameCount = json.value("frameCount", 0);
// Restaurer tanks
tanks.clear();
if (json.contains("tanks") && json["tanks"].is_array()) {
for (const auto& tankJson : json["tanks"]) {
Tank tank;
tank.id = tankJson.value("id", 0);
tank.x = tankJson.value("x", 0.0f);
tank.y = tankJson.value("y", 0.0f);
tank.vx = tankJson.value("vx", 0.0f);
tank.vy = tankJson.value("vy", 0.0f);
tank.cooldown = tankJson.value("cooldown", 0.0f);
tank.targetX = tankJson.value("targetX", 0.0f);
tank.targetY = tankJson.value("targetY", 0.0f);
tanks.push_back(tank);
}
}
logger->info("State restored: {} tanks, frame {}", tanks.size(), frameCount);
}
void TankModule::updateTank(Tank& tank, float dt) {
// Update position
tank.x += tank.vx * dt;
tank.y += tank.vy * dt;
// Bounce sur les bords (map 100x100)
if (tank.x < 0.0f || tank.x > 100.0f) {
tank.vx = -tank.vx;
tank.x = std::clamp(tank.x, 0.0f, 100.0f);
}
if (tank.y < 0.0f || tank.y > 100.0f) {
tank.vy = -tank.vy;
tank.y = std::clamp(tank.y, 0.0f, 100.0f);
}
// Update cooldown
if (tank.cooldown > 0.0f) {
tank.cooldown -= dt;
}
// Déplacer vers target
float dx = tank.targetX - tank.x;
float dy = tank.targetY - tank.y;
float dist = std::sqrt(dx * dx + dy * dy);
if (dist > 0.1f) {
// Normaliser et appliquer velocity
float speed = std::sqrt(tank.vx * tank.vx + tank.vy * tank.vy);
tank.vx = (dx / dist) * speed;
tank.vy = (dy / dist) * speed;
}
}
void TankModule::spawnTanks(int count) {
std::mt19937 rng(42); // Seed fixe pour reproductibilité
std::uniform_real_distribution<float> posDist(0.0f, 100.0f);
std::uniform_real_distribution<float> velDist(-5.0f, 5.0f);
std::uniform_real_distribution<float> cooldownDist(0.0f, 5.0f);
for (int i = 0; i < count; i++) {
Tank tank;
tank.id = i;
tank.x = posDist(rng);
tank.y = posDist(rng);
tank.vx = velDist(rng);
tank.vy = velDist(rng);
tank.cooldown = cooldownDist(rng);
tank.targetX = posDist(rng);
tank.targetY = posDist(rng);
tanks.push_back(tank);
}
logger->debug("Spawned {} tanks", count);
}
} // namespace grove
// Export symbols
extern "C" {
grove::IModule* createModule() {
return new grove::TankModule();
}
void destroyModule(grove::IModule* module) {
delete module;
}
}

48
tests/modules/TankModule.h Normal file
View File

@ -0,0 +1,48 @@
#pragma once
#include "grove/IModule.h"
#include "grove/IDataNode.h"
#include <vector>
#include <random>
#include <memory>
#include <spdlog/spdlog.h>
namespace grove {
class TankModule : public IModule {
public:
struct Tank {
float x, y; // Position
float vx, vy; // Vélocité
float cooldown; // Temps avant prochain tir
float targetX, targetY; // Destination
int id; // Identifiant unique
};
// IModule interface
void process(const IDataNode& input) override;
void setConfiguration(const IDataNode& configNode, IIO* io, ITaskScheduler* scheduler) override;
const IDataNode& getConfiguration() override;
std::unique_ptr<IDataNode> getHealthStatus() override;
void shutdown() override;
std::unique_ptr<IDataNode> getState() override;
void setState(const IDataNode& state) override;
std::string getType() const override;
bool isIdle() const override { return true; }
private:
std::vector<Tank> tanks;
int frameCount = 0;
std::string moduleVersion = "v1.0";std::shared_ptr<spdlog::logger> logger;
std::unique_ptr<IDataNode> config;
void updateTank(Tank& tank, float dt);
void spawnTanks(int count);
};
} // namespace grove
// Export symbols
extern "C" {
grove::IModule* createModule();
void destroyModule(grove::IModule* module);
}

11
tests/modules/TestModule.cpp
View File

@ -17,7 +17,7 @@ namespace grove {
class TestModule : public IModule { class TestModule : public IModule {
private: private:
int counter = 0; int counter = 0;
std::string moduleVersion = "v1.0"; std::string moduleVersion = "v2.0 RELOADED";
IIO* io = nullptr; IIO* io = nullptr;
ITaskScheduler* scheduler = nullptr; ITaskScheduler* scheduler = nullptr;
std::unique_ptr<IDataNode> config; std::unique_ptr<IDataNode> config;
@ -52,8 +52,8 @@ public:
// Clone configuration for storage // Clone configuration for storage
config = std::make_unique<JsonDataNode>("config", nlohmann::json::object()); config = std::make_unique<JsonDataNode>("config", nlohmann::json::object());
// Extract version if available // Extract version if available (use current moduleVersion as default)
moduleVersion = configNode.getString("version", "v1.0"); moduleVersion = configNode.getString("version", moduleVersion);
std::cout << "[TestModule] Version set to: " << moduleVersion << std::endl; std::cout << "[TestModule] Version set to: " << moduleVersion << std::endl;
} }
@ -99,6 +99,11 @@ public:
std::string getType() const override { std::string getType() const override {
return "TestModule"; return "TestModule";
} }
bool isIdle() const override {
// TestModule has no async operations, always idle
return true;
}
}; };
} // namespace grove } // namespace grove