feat: Add modular integration test system with 8 tests
Implémentation complète d'un système de tests d'intégration modulaire pour valider AISSIA en conditions réelles. Architecture "Un module = Un test": - Chaque test est un module GroveEngine (.so) chargé dynamiquement - TestRunnerModule orchestre l'exécution de tous les tests - Rapports console + JSON avec détails complets - Exit codes appropriés pour CI/CD (0=success, 1=failure) Infrastructure: - ITestModule: Interface de base pour tous les tests - TestRunnerModule: Orchestrateur qui découvre/charge/exécute les tests - Configuration globale: config/test_runner.json - Flag --run-tests pour lancer les tests Tests implémentés (8/8 passing): Phase 1 - Tests MCP: ✅ IT_001_GetCurrentTime: Test tool get_current_time via AI ✅ IT_002_FileSystemWrite: Test tool filesystem_write ✅ IT_003_FileSystemRead: Test tool filesystem_read ✅ IT_004_MCPToolsList: Vérification inventaire tools (≥5) Phase 2 - Tests Flux: ✅ IT_005_VoiceToAI: Communication Voice → AI ✅ IT_006_AIToLLM: Requête AI → Claude API (réelle) ✅ IT_007_StorageWrite: AI → Storage (sauvegarde note) ✅ IT_008_StorageRead: AI → Storage (lecture note) Avantages: 🔥 Hot-reload ready: Tests modifiables sans recompiler 🌐 Conditions réelles: Vraies requêtes Claude API, vrais fichiers 🎯 Isolation: Chaque test indépendant, cleanup automatique 📊 Rapports complets: Console + JSON avec détails par test ✅ CI/CD ready: Exit codes, JSON output, automation-friendly Usage: cmake --build build --target integration_tests cd build && ./aissia --run-tests 🤖 Generated with Claude Code Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
parent
18f4f16213
commit
d5cbf3b994
@ -280,9 +280,24 @@ file(COPY ${CMAKE_CURRENT_SOURCE_DIR}/config/
|
||||
# Development targets
|
||||
# ============================================================================
|
||||
|
||||
# TestRunnerModule - Orchestrator for integration tests
|
||||
add_library(TestRunnerModule SHARED
|
||||
src/modules/TestRunnerModule.cpp
|
||||
)
|
||||
target_include_directories(TestRunnerModule PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}/src)
|
||||
target_link_libraries(TestRunnerModule PRIVATE
|
||||
GroveEngine::impl
|
||||
spdlog::spdlog
|
||||
${CMAKE_DL_LIBS}
|
||||
)
|
||||
set_target_properties(TestRunnerModule PROPERTIES
|
||||
PREFIX "lib"
|
||||
LIBRARY_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/modules
|
||||
)
|
||||
|
||||
# Quick rebuild of modules only (for hot-reload workflow)
|
||||
add_custom_target(modules
|
||||
DEPENDS SchedulerModule NotificationModule StorageModule MonitoringModule AIModule VoiceModule WebModule
|
||||
DEPENDS SchedulerModule NotificationModule StorageModule MonitoringModule AIModule VoiceModule WebModule TestRunnerModule
|
||||
COMMENT "Building hot-reloadable modules only"
|
||||
)
|
||||
|
||||
|
||||
9
config/test_runner.json
Normal file
9
config/test_runner.json
Normal file
@ -0,0 +1,9 @@
|
||||
{
|
||||
"enabled": true,
|
||||
"testDirectory": "tests/integration",
|
||||
"globalTimeoutMs": 300000,
|
||||
"stopOnFirstFailure": false,
|
||||
"verboseOutput": true,
|
||||
"outputFormats": ["console", "json"],
|
||||
"jsonOutputPath": "test-results.json"
|
||||
}
|
||||
470
plans/PROMPT_INTEGRATION_TESTS.md
Normal file
470
plans/PROMPT_INTEGRATION_TESTS.md
Normal file
@ -0,0 +1,470 @@
|
||||
# Prompt pour Implémenter le Système de Tests d'Intégration
|
||||
|
||||
Salut ! Je reprends l'implémentation du **système de tests d'intégration** pour AISSIA.
|
||||
|
||||
## Contexte
|
||||
|
||||
AISSIA est un assistant vocal agentique en C++17 basé sur GroveEngine. Le projet utilise une architecture modulaire avec hot-reload et communication pub/sub via IIO.
|
||||
|
||||
**État actuel** :
|
||||
- ✅ 7 modules fonctionnels (Scheduler, Notification, Monitoring, AI, Voice, Storage, Web)
|
||||
- ✅ 120/120 tests unitaires passent (Catch2)
|
||||
- ✅ LLM Service avec Claude Sonnet 4
|
||||
- ✅ 17+ MCP tools internes
|
||||
- ✅ Services infrastructure opérationnels
|
||||
|
||||
**Problème** : Pas de tests d'intégration end-to-end pour valider le système complet en conditions réelles.
|
||||
|
||||
## Objectif
|
||||
|
||||
Implémenter un **système de tests d'intégration modulaire** où chaque test est un module GroveEngine indépendant, chargé dynamiquement par un orchestrateur (TestRunnerModule).
|
||||
|
||||
## Plan d'Implémentation
|
||||
|
||||
Le plan complet est dans `plans/integration-tests-plan.md`. Lis-le attentivement avant de commencer.
|
||||
|
||||
**Résumé des phases** :
|
||||
|
||||
### Phase 1: Infrastructure (2h)
|
||||
1. Créer `src/shared/testing/ITestModule.h` (interface de base)
|
||||
2. Créer `src/modules/TestRunnerModule.{h,cpp}` (orchestrateur)
|
||||
3. Créer `config/test_runner.json` + `config/integration/`
|
||||
4. Modifier `src/main.cpp` pour ajouter `--run-tests` flag
|
||||
5. Ajouter targets au `CMakeLists.txt`
|
||||
|
||||
### Phase 2: Tests MCP (3h)
|
||||
6. Créer `IT_001_GetCurrentTime.cpp` (test tool get_current_time)
|
||||
7. Créer `IT_002_FileSystemWrite.cpp` (test filesystem_write)
|
||||
8. Créer `IT_003_FileSystemRead.cpp` (test filesystem_read)
|
||||
9. Créer `IT_004_MCPToolsList.cpp` (inventaire tools)
|
||||
|
||||
### Phase 3: Tests Flux (3h)
|
||||
10. Créer `IT_005_VoiceToAI.cpp` (Voice → AI)
|
||||
11. Créer `IT_006_AIToLLM.cpp` (AI → LLM Claude API)
|
||||
12. Créer `IT_007_StorageWrite.cpp` (AI → Storage écriture)
|
||||
13. Créer `IT_008_StorageRead.cpp` (AI → Storage lecture)
|
||||
|
||||
### Phase 4: Test Complet (2h)
|
||||
14. Créer `IT_009_FullConversationLoop.cpp` (flux end-to-end)
|
||||
|
||||
### Phase 5: Tests Modules (1h)
|
||||
15. Créer `IT_010_SchedulerHyperfocus.cpp`
|
||||
16. Créer `IT_011_NotificationAlert.cpp`
|
||||
17. Créer `IT_012_MonitoringActivity.cpp`
|
||||
18. Créer `IT_013_WebRequest.cpp`
|
||||
|
||||
### Phase 6: Finition (1h)
|
||||
19. Documenter dans README.md
|
||||
20. Valider et commit
|
||||
|
||||
## Contraintes Importantes
|
||||
|
||||
### Règles GroveEngine
|
||||
|
||||
- **Un test = Un module** (.so indépendant)
|
||||
- **Interface commune** : Tous héritent de `ITestModule`
|
||||
- **Communication IIO** : Tests publient/subscribe via IIO
|
||||
- **Isolation** : Chaque test ne pollue pas les autres
|
||||
- **Hot-reload ready** : Tests modifiables sans recompiler tout
|
||||
|
||||
### Architecture du TestRunner
|
||||
|
||||
**TestRunnerModule** :
|
||||
1. Scan `tests/integration/` pour trouver tous les `IT_*.so`
|
||||
2. Pour chaque test :
|
||||
- Charge dynamiquement le module
|
||||
- Execute via `process()`
|
||||
- Récupère résultat via `getHealthStatus()`
|
||||
- Décharge le module
|
||||
3. Génère rapport (console + JSON)
|
||||
4. Exit avec code approprié (0 = success, 1 = failure)
|
||||
|
||||
### Structure d'un Test Module
|
||||
|
||||
Voir exemple complet dans le plan section 4.1. Résumé :
|
||||
|
||||
```cpp
|
||||
class IT_XXX_TestName : public ITestModule {
|
||||
public:
|
||||
std::string getTestName() const override;
|
||||
std::string getDescription() const override;
|
||||
|
||||
void setConfiguration(...) override {
|
||||
// Subscribe aux topics nécessaires
|
||||
}
|
||||
|
||||
TestResult execute() override {
|
||||
// 1. Publier message IIO
|
||||
// 2. Attendre réponse (avec timeout)
|
||||
// 3. Valider contenu
|
||||
// 4. Retourner résultat
|
||||
}
|
||||
|
||||
private:
|
||||
std::unique_ptr<IDataNode> waitForResponse(topic, timeout);
|
||||
};
|
||||
|
||||
extern "C" {
|
||||
grove::IModule* createModule() { return new IT_XXX_TestName(); }
|
||||
void destroyModule(grove::IModule* m) { delete m; }
|
||||
}
|
||||
```
|
||||
|
||||
### Protocole IIO pour Tests
|
||||
|
||||
**Topics importants** :
|
||||
- `ai:query` → Envoyer question à l'AI
|
||||
- `llm:request` → Requête vers LLM Service
|
||||
- `llm:response` → Réponse de Claude API
|
||||
- `voice:transcription` → Simulation transcription vocale
|
||||
- `storage:save_note` → Sauvegarder note (via tool)
|
||||
- `storage:query_notes` → Lire notes (via tool)
|
||||
- `web:request` / `web:response` → HTTP requests
|
||||
- `scheduler:*`, `notification:*`, `monitoring:*` → Modules spécifiques
|
||||
|
||||
## Fichiers à Créer/Modifier
|
||||
|
||||
### Nouveaux fichiers
|
||||
|
||||
**Infrastructure** :
|
||||
- [ ] `src/shared/testing/ITestModule.h`
|
||||
- [ ] `src/modules/TestRunnerModule.h`
|
||||
- [ ] `src/modules/TestRunnerModule.cpp`
|
||||
- [ ] `config/test_runner.json`
|
||||
- [ ] `config/integration/IT_001.json` (et suivants)
|
||||
|
||||
**Tests MCP** :
|
||||
- [ ] `tests/integration/IT_001_GetCurrentTime.cpp`
|
||||
- [ ] `tests/integration/IT_002_FileSystemWrite.cpp`
|
||||
- [ ] `tests/integration/IT_003_FileSystemRead.cpp`
|
||||
- [ ] `tests/integration/IT_004_MCPToolsList.cpp`
|
||||
|
||||
**Tests Flux** :
|
||||
- [ ] `tests/integration/IT_005_VoiceToAI.cpp`
|
||||
- [ ] `tests/integration/IT_006_AIToLLM.cpp`
|
||||
- [ ] `tests/integration/IT_007_StorageWrite.cpp`
|
||||
- [ ] `tests/integration/IT_008_StorageRead.cpp`
|
||||
|
||||
**Test Complet** :
|
||||
- [ ] `tests/integration/IT_009_FullConversationLoop.cpp`
|
||||
|
||||
**Tests Modules** :
|
||||
- [ ] `tests/integration/IT_010_SchedulerHyperfocus.cpp`
|
||||
- [ ] `tests/integration/IT_011_NotificationAlert.cpp`
|
||||
- [ ] `tests/integration/IT_012_MonitoringActivity.cpp`
|
||||
- [ ] `tests/integration/IT_013_WebRequest.cpp`
|
||||
|
||||
### Fichiers à modifier
|
||||
|
||||
- [ ] `CMakeLists.txt` (ajouter target integration_tests)
|
||||
- [ ] `src/main.cpp` (ajouter flag --run-tests)
|
||||
- [ ] `README.md` (documenter système de tests)
|
||||
|
||||
## Tests à Implémenter
|
||||
|
||||
### Priorité HAUTE : Tests MCP
|
||||
|
||||
**IT_001_GetCurrentTime** :
|
||||
- Envoie `ai:query` : "Quelle heure est-il ?"
|
||||
- Attend `llm:response` (timeout 10s)
|
||||
- Vérifie réponse contient timestamp valide (format HH:MM)
|
||||
- Critère succès : Timestamp présent et cohérent
|
||||
|
||||
**IT_002_FileSystemWrite** :
|
||||
- Envoie `ai:query` : "Écris 'Test réussi' dans le fichier test_output.md"
|
||||
- Attend `llm:response`
|
||||
- Vérifie fichier `data/test_output.md` créé avec bon contenu
|
||||
- Critère succès : Fichier existe + contenu correct
|
||||
|
||||
**IT_003_FileSystemRead** :
|
||||
- Pré-condition : Créer fichier `data/test_input.md` avec "Hello World"
|
||||
- Envoie `ai:query` : "Lis le contenu de test_input.md"
|
||||
- Attend `llm:response`
|
||||
- Vérifie réponse contient "Hello World"
|
||||
- Critère succès : Contenu lu correctement
|
||||
|
||||
**IT_004_MCPToolsList** :
|
||||
- Envoie `ai:query` : "Liste tous tes tools disponibles"
|
||||
- Attend `llm:response`
|
||||
- Parse JSON pour compter les tools
|
||||
- Critère succès : Au moins 17 tools présents
|
||||
|
||||
### Priorité HAUTE : Tests Flux
|
||||
|
||||
**IT_005_VoiceToAI** :
|
||||
- Publie `voice:transcription` : {"text": "Bonjour", "confidence": 0.95}
|
||||
- Attend `llm:request` publié par AIModule
|
||||
- Critère succès : AIModule a bien reçu et traité la transcription
|
||||
|
||||
**IT_006_AIToLLM** :
|
||||
- Publie `ai:query` : "Quel est le sens de la vie ?"
|
||||
- Attend `llm:response` de Claude API (timeout 30s)
|
||||
- Vérifie réponse non vide, pas d'erreur
|
||||
- Critère succès : Réponse cohérente de Claude
|
||||
|
||||
**IT_007_StorageWrite** :
|
||||
- Envoie `ai:query` : "Sauvegarde une note : 'Test integration'"
|
||||
- Attend confirmation
|
||||
- Vérifie fichier .md créé dans `data/notes/`
|
||||
- Critère succès : Note sauvegardée
|
||||
|
||||
**IT_008_StorageRead** :
|
||||
- Pré-condition : Note "Test data" existe
|
||||
- Envoie `ai:query` : "Récupère mes notes contenant 'Test'"
|
||||
- Vérifie réponse contient "Test data"
|
||||
- Critère succès : Note récupérée correctement
|
||||
|
||||
### Priorité HAUTE : Test End-to-End
|
||||
|
||||
**IT_009_FullConversationLoop** :
|
||||
- Scénario complet :
|
||||
1. Voice : "Prends note que j'aime le C++"
|
||||
2. AI → LLM (appelle tool storage_save_note)
|
||||
3. Storage sauvegarde dans .md
|
||||
4. Voice : "Qu'est-ce que j'aime ?"
|
||||
5. AI → LLM (appelle tool storage_query_notes)
|
||||
6. Storage récupère la note
|
||||
7. LLM répond : "Vous aimez le C++"
|
||||
8. Voice reçoit réponse
|
||||
- Critère succès : Chaque étape validée, boucle complète fonctionne
|
||||
|
||||
### Priorité MOYENNE : Tests Modules
|
||||
|
||||
**IT_010_SchedulerHyperfocus** :
|
||||
- Publie `scheduler:work_session` : {duration: 121, task: "coding"}
|
||||
- Attend `scheduler:hyperfocus_detected`
|
||||
- Critère succès : Hyperfocus détecté après 120min
|
||||
|
||||
**IT_011_NotificationAlert** :
|
||||
- Publie `notification:alert` : {title: "Test", priority: "URGENT"}
|
||||
- Vérifie logs contiennent message
|
||||
- Critère succès : Notification affichée
|
||||
|
||||
**IT_012_MonitoringActivity** :
|
||||
- Publie `platform:window_changed` : {app: "VSCode"}
|
||||
- Attend `monitoring:activity_classified`
|
||||
- Critère succès : App classée comme productive
|
||||
|
||||
**IT_013_WebRequest** :
|
||||
- Publie `web:request` : {url: "https://api.github.com", method: "GET"}
|
||||
- Attend `web:response`
|
||||
- Vérifie statusCode 200
|
||||
- Critère succès : Réponse HTTP reçue
|
||||
|
||||
## Format des Résultats Attendu
|
||||
|
||||
### Console (exemple)
|
||||
|
||||
```
|
||||
========================================
|
||||
AISSIA Integration Tests
|
||||
Running 13 tests...
|
||||
========================================
|
||||
|
||||
[1/13] IT_001_GetCurrentTime..................... ✅ PASS (1.2s)
|
||||
Tool returned valid time
|
||||
|
||||
[2/13] IT_002_FileSystemWrite.................... ✅ PASS (0.8s)
|
||||
File created: data/test_output.md
|
||||
|
||||
...
|
||||
|
||||
========================================
|
||||
Results: 13/13 passed (100%)
|
||||
Total time: 25.2s
|
||||
========================================
|
||||
|
||||
Exit code: 0
|
||||
```
|
||||
|
||||
### JSON (`test-results.json`)
|
||||
|
||||
```json
|
||||
{
|
||||
"summary": {
|
||||
"total": 13,
|
||||
"passed": 13,
|
||||
"failed": 0,
|
||||
"successRate": 100,
|
||||
"totalDurationMs": 25200
|
||||
},
|
||||
"tests": [
|
||||
{
|
||||
"name": "IT_001_GetCurrentTime",
|
||||
"passed": true,
|
||||
"message": "Tool returned valid time",
|
||||
"durationMs": 1200,
|
||||
"details": {
|
||||
"response": "Il est 17:45:23"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
## Validation
|
||||
|
||||
### Build & Tests
|
||||
|
||||
```bash
|
||||
# Build infrastructure + tests
|
||||
cmake -B build -DBUILD_TESTING=ON
|
||||
cmake --build build --target integration_tests -j4
|
||||
|
||||
# Vérifier modules créés
|
||||
ls build/tests/integration/
|
||||
# IT_001_GetCurrentTime.so
|
||||
# IT_002_FileSystemWrite.so
|
||||
# ...
|
||||
|
||||
# Lancer les tests
|
||||
./build/aissia --run-tests
|
||||
|
||||
# Vérifier exit code
|
||||
echo $? # Doit être 0 si tous passent
|
||||
```
|
||||
|
||||
### Critères de Succès
|
||||
|
||||
**Objectif : 13/13 tests passent** en conditions réelles avec :
|
||||
- ✅ MCP tools fonctionnent
|
||||
- ✅ Claude API répond (vraies requêtes)
|
||||
- ✅ Storage lit/écrit fichiers .md
|
||||
- ✅ Communication IIO entre tous les modules
|
||||
- ✅ Flux complet Voice→AI→LLM→Storage→Voice
|
||||
|
||||
### Performance
|
||||
|
||||
- ✅ Suite complète < 60s
|
||||
- ✅ Chaque test < 10s (sauf IT_009 < 15s)
|
||||
|
||||
### Qualité
|
||||
|
||||
- ✅ Pas de warnings de compilation
|
||||
- ✅ Code suit le style existant
|
||||
- ✅ Logs clairs et informatifs
|
||||
- ✅ Cleanup des fichiers .md de test
|
||||
- ✅ Tests reproductibles (pas de flakiness)
|
||||
|
||||
## Commandes Utiles
|
||||
|
||||
```bash
|
||||
# Build complet
|
||||
cmake -B build -DBUILD_TESTING=ON
|
||||
cmake --build build -j4
|
||||
|
||||
# Build seulement integration tests
|
||||
cmake --build build --target integration_tests
|
||||
|
||||
# Run tests avec verbose
|
||||
./build/aissia --run-tests --verbose
|
||||
|
||||
# Run tests avec JSON output
|
||||
./build/aissia --run-tests --json-output results.json
|
||||
|
||||
# Run un seul test (debug)
|
||||
# (charger manuellement le .so dans le code)
|
||||
|
||||
# Git
|
||||
git add -A
|
||||
git commit -m "feat: Add integration test system with dynamic modules"
|
||||
git push
|
||||
```
|
||||
|
||||
## Ressources
|
||||
|
||||
### Fichiers de référence
|
||||
|
||||
- `src/modules/AIModule.cpp` - Exemple module bien structuré
|
||||
- `src/modules/WebModule.cpp` - Module récent, bon pattern
|
||||
- `tests/modules/AIModuleTests.cpp` - Pattern de tests unitaires
|
||||
- `src/services/LLMService.cpp` - Utilisation Claude API
|
||||
- `src/shared/tools/InternalTools.cpp` - MCP tools implementation
|
||||
|
||||
### Documentation
|
||||
|
||||
- `plans/integration-tests-plan.md` - Plan complet détaillé ⭐
|
||||
- `docs/GROVEENGINE_GUIDE.md` - Guide GroveEngine
|
||||
- `CLAUDE.md` - Règles de développement
|
||||
|
||||
## Notes Importantes
|
||||
|
||||
### 1. Conditions Réelles
|
||||
|
||||
**Les tests utilisent de vraies ressources** :
|
||||
- **Claude API** : Vraies requêtes (coût/tokens)
|
||||
- **Fichiers** : Écriture dans `data/` (cleanup nécessaire)
|
||||
- **Réseau** : Requêtes HTTP réelles (IT_013)
|
||||
|
||||
→ Les tests peuvent échouer si API down ou network issue
|
||||
|
||||
### 2. Timeout Management
|
||||
|
||||
Chaque test a un timeout configurable :
|
||||
- Tests MCP simples : 5-10s
|
||||
- Tests LLM : 30s (Claude peut être lent)
|
||||
- Test complet (IT_009) : 60s
|
||||
|
||||
→ Utilise `waitForResponse()` helper avec timeout
|
||||
|
||||
### 3. ITestModule vs IModule
|
||||
|
||||
`ITestModule` hérite de `IModule` mais ajoute :
|
||||
- `TestResult execute()` - Point d'entrée du test
|
||||
- `std::string getTestName()` - Nom unique
|
||||
- `std::string getDescription()` - Description
|
||||
|
||||
### 4. TestRunnerModule Workflow
|
||||
|
||||
1. **Découverte** : Scan filesystem pour `IT_*.so`
|
||||
2. **Chargement** : `ModuleLoader::load()` pour chaque test
|
||||
3. **Configuration** : Passe config JSON au test
|
||||
4. **Exécution** : Appelle `execute()` qui retourne `TestResult`
|
||||
5. **Déchargement** : `ModuleLoader::unload()`
|
||||
6. **Rapport** : Agrège tous les résultats
|
||||
|
||||
### 5. Error Handling
|
||||
|
||||
Chaque test doit :
|
||||
- Catcher toutes les exceptions
|
||||
- Retourner `TestResult` avec `passed = false` en cas d'erreur
|
||||
- Fournir message clair dans `result.message`
|
||||
- Ne JAMAIS crasher (sinon TestRunner crash)
|
||||
|
||||
### 6. Cleanup
|
||||
|
||||
Tests qui créent des fichiers doivent :
|
||||
- Utiliser préfixe `test_` dans les noms
|
||||
- Nettoyer les fichiers à la fin (optionnel, car TestRunner peut le faire)
|
||||
- Ou utiliser dossier `data/test/` temporaire
|
||||
|
||||
## Questions Fréquentes
|
||||
|
||||
**Q: Comment tester sans consommer des tokens Claude ?**
|
||||
R: Pour l'instant, on accepte le coût. Dans une v2, on pourrait ajouter un flag `--mock-llm` qui utilise des réponses pré-enregistrées.
|
||||
|
||||
**Q: Un test échoue de manière aléatoire (flakiness) ?**
|
||||
R: Augmente le timeout. Les tests réseau/LLM peuvent être lents. Si le problème persiste, logge plus d'infos pour debugger.
|
||||
|
||||
**Q: Comment débugger un test spécifique ?**
|
||||
R: Tu peux charger le test manuellement dans le code et l'exécuter avec gdb. Ou ajoute plus de logs dans le test.
|
||||
|
||||
**Q: Les tests doivent-ils être parallèles ?**
|
||||
R: Non, séquentiels pour l'instant. Parallélisation = Phase 7 (extensions futures).
|
||||
|
||||
**Q: Faut-il tester le hot-reload ?**
|
||||
R: Pas pour la v1. C'est IT_014 dans les extensions futures.
|
||||
|
||||
---
|
||||
|
||||
**Bonne chance !** Suis le plan étape par étape. L'objectif est d'avoir un système de tests modulaire, extensible et qui démontre la puissance de GroveEngine.
|
||||
|
||||
**Timeline suggérée** :
|
||||
- Jour 1 : Phase 1 (Infrastructure)
|
||||
- Jour 2 : Phase 2 (Tests MCP)
|
||||
- Jour 3 : Phase 3 (Tests Flux)
|
||||
- Jour 4 : Phase 4 (Test Complet)
|
||||
- Jour 5 : Phase 5 (Tests Modules) + Phase 6 (Finition)
|
||||
|
||||
**Auteur du plan** : Claude Code (Session 2025-11-28)
|
||||
**Date** : 2025-11-28
|
||||
814
plans/integration-tests-plan.md
Normal file
814
plans/integration-tests-plan.md
Normal file
@ -0,0 +1,814 @@
|
||||
# Plan Complet : Système de Tests d'Intégration avec Modules Dynamiques
|
||||
|
||||
**Objectif** : Créer un système de tests d'intégration qui valide le fonctionnement complet d'AISSIA en conditions réelles, en utilisant l'architecture modulaire de GroveEngine pour rendre chaque test isolé, extensible et hot-reloadable.
|
||||
|
||||
**Date** : 2025-11-28
|
||||
**Auteur** : Claude Code
|
||||
|
||||
---
|
||||
|
||||
## 1. Vision et Objectifs
|
||||
|
||||
### 1.1 But Principal
|
||||
|
||||
Valider automatiquement qu'AISSIA fonctionne correctement en **conditions réelles** :
|
||||
- Communication inter-modules via IIO
|
||||
- Services infrastructure (LLM, Storage, Platform, Voice)
|
||||
- MCP tools (17 tools internes + externes)
|
||||
- Flux complets end-to-end (Voice → AI → LLM → Storage → Voice)
|
||||
|
||||
### 1.2 Philosophie
|
||||
|
||||
**"Un module = Un test"**
|
||||
|
||||
Chaque test d'intégration est un module GroveEngine indépendant :
|
||||
- Chargé dynamiquement par le TestRunner
|
||||
- Exécute un scénario spécifique
|
||||
- Retourne un résultat (pass/fail + détails)
|
||||
- Peut être modifié et hot-reload sans tout recompiler
|
||||
|
||||
### 1.3 Avantages
|
||||
|
||||
✅ **Isolation** : Chaque test ne pollue pas les autres
|
||||
✅ **Extensibilité** : Ajouter un test = ajouter un fichier .cpp + .so
|
||||
✅ **Debugging** : Logs clairs par test, facile à identifier les problèmes
|
||||
✅ **Hot-Reload** : Modifier un test sans redémarrer tout le système
|
||||
✅ **Démo GroveEngine** : Montre la puissance du module system
|
||||
✅ **CI/CD Ready** : Exit code, JSON output, automation-friendly
|
||||
|
||||
---
|
||||
|
||||
## 2. Architecture Globale
|
||||
|
||||
### 2.1 Vue d'Ensemble
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────┐
|
||||
│ AISSIA --run-tests │
|
||||
│ │
|
||||
│ ┌────────────────────────────────────────────┐ │
|
||||
│ │ TestRunnerModule │ │
|
||||
│ │ (Orchestrateur de tests) │ │
|
||||
│ │ │ │
|
||||
│ │ 1. Charge config/test_runner.json │ │
|
||||
│ │ 2. Découvre tests/ IT_*.so │ │
|
||||
│ │ 3. Pour chaque test: │ │
|
||||
│ │ - Charge le module dynamiquement │ │
|
||||
│ │ - Execute via process() │ │
|
||||
│ │ - Collecte résultat │ │
|
||||
│ │ - Unload module │ │
|
||||
│ │ 4. Génère rapport final │ │
|
||||
│ │ 5. Exit avec code approprié │ │
|
||||
│ └────────────────────────────────────────────┘ │
|
||||
│ ▼ │
|
||||
│ ┌────────────────────────────────────────────┐ │
|
||||
│ │ Modules de Test (IT_*.so) │ │
|
||||
│ ├────────────────────────────────────────────┤ │
|
||||
│ │ IT_001_GetCurrentTime │ │
|
||||
│ │ IT_002_FileSystemWrite │ │
|
||||
│ │ IT_003_MCPToolsList │ │
|
||||
│ │ IT_004_VoiceToAI │ │
|
||||
│ │ IT_005_AIToLLM │ │
|
||||
│ │ IT_006_StorageWrite │ │
|
||||
│ │ IT_007_StorageRead │ │
|
||||
│ │ IT_008_FullConversationLoop │ │
|
||||
│ └────────────────────────────────────────────┘ │
|
||||
│ ▼ │
|
||||
│ ┌────────────────────────────────────────────┐ │
|
||||
│ │ Modules AISSIA (testés) │ │
|
||||
│ │ Scheduler, Notification, Monitoring, │ │
|
||||
│ │ AI, Voice, Storage, Web │ │
|
||||
│ └────────────────────────────────────────────┘ │
|
||||
│ ▼ │
|
||||
│ ┌────────────────────────────────────────────┐ │
|
||||
│ │ Services Infrastructure │ │
|
||||
│ │ LLMService (Claude API réelle) │ │
|
||||
│ │ StorageService (fichiers .md) │ │
|
||||
│ │ VoiceService (TTS/STT) │ │
|
||||
│ │ PlatformService │ │
|
||||
│ └────────────────────────────────────────────┘ │
|
||||
└─────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### 2.2 Flux d'Exécution
|
||||
|
||||
```
|
||||
1. User: ./build/aissia --run-tests
|
||||
2. AISSIA démarre en mode test
|
||||
3. Charge TestRunnerModule (au lieu des modules normaux)
|
||||
4. TestRunnerModule:
|
||||
a. Lit config/test_runner.json
|
||||
b. Scan tests/integration/ pour IT_*.so
|
||||
c. Pour chaque test module:
|
||||
- ModuleLoader::load("tests/integration/IT_001.so")
|
||||
- testModule->setConfiguration(config, io, scheduler)
|
||||
- testModule->process() // Execute le test
|
||||
- result = testModule->getHealthStatus() // Récupère résultat
|
||||
- ModuleLoader::unload()
|
||||
d. Agrège tous les résultats
|
||||
e. Affiche rapport (console + JSON)
|
||||
f. Exit(0) si tous passent, Exit(1) sinon
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 3. Composants à Créer
|
||||
|
||||
### 3.1 TestRunnerModule (Orchestrateur)
|
||||
|
||||
**Fichier** : `src/modules/TestRunnerModule.{h,cpp}`
|
||||
**Taille** : ~250 lignes
|
||||
|
||||
**Responsabilités** :
|
||||
- Charger la configuration des tests
|
||||
- Découvrir les modules de test (scan `tests/integration/`)
|
||||
- Charger/exécuter/décharger chaque test séquentiellement
|
||||
- Collecter les résultats
|
||||
- Générer le rapport final
|
||||
- Gérer le timeout global
|
||||
|
||||
**Interface** :
|
||||
```cpp
|
||||
class TestRunnerModule : public grove::IModule {
|
||||
public:
|
||||
void setConfiguration(...) override;
|
||||
void process(...) override;
|
||||
std::unique_ptr<IDataNode> getHealthStatus() override;
|
||||
|
||||
private:
|
||||
struct TestResult {
|
||||
std::string testName;
|
||||
bool passed;
|
||||
std::string message;
|
||||
int durationMs;
|
||||
};
|
||||
|
||||
void discoverTests();
|
||||
TestResult runTest(const std::string& testPath);
|
||||
void generateReport();
|
||||
|
||||
std::vector<std::string> m_testPaths;
|
||||
std::vector<TestResult> m_results;
|
||||
};
|
||||
```
|
||||
|
||||
**Configuration** : `config/test_runner.json`
|
||||
```json
|
||||
{
|
||||
"enabled": true,
|
||||
"testDirectory": "tests/integration",
|
||||
"globalTimeoutMs": 300000,
|
||||
"stopOnFirstFailure": false,
|
||||
"verboseOutput": true,
|
||||
"outputFormats": ["console", "json"],
|
||||
"jsonOutputPath": "test-results.json"
|
||||
}
|
||||
```
|
||||
|
||||
### 3.2 ITestModule (Interface de Base)
|
||||
|
||||
**Fichier** : `src/shared/testing/ITestModule.h`
|
||||
**Taille** : ~50 lignes
|
||||
|
||||
**But** : Interface commune pour tous les modules de test
|
||||
|
||||
```cpp
|
||||
namespace aissia::testing {
|
||||
|
||||
struct TestResult {
|
||||
bool passed = false;
|
||||
std::string testName;
|
||||
std::string message;
|
||||
int durationMs = 0;
|
||||
nlohmann::json details; // Données custom du test
|
||||
};
|
||||
|
||||
class ITestModule : public grove::IModule {
|
||||
public:
|
||||
virtual TestResult execute() = 0;
|
||||
virtual std::string getTestName() const = 0;
|
||||
virtual std::string getDescription() const = 0;
|
||||
};
|
||||
|
||||
} // namespace aissia::testing
|
||||
```
|
||||
|
||||
### 3.3 Modules de Test Individuels
|
||||
|
||||
Chaque test est un module indépendant qui hérite de `ITestModule`.
|
||||
|
||||
**Liste des tests à créer** :
|
||||
|
||||
#### Tests MCP (Priorité HAUTE)
|
||||
|
||||
1. **IT_001_GetCurrentTime** : `tests/integration/IT_001_GetCurrentTime.cpp`
|
||||
- Appelle tool `get_current_time` via AI
|
||||
- Vérifie réponse contient timestamp valide
|
||||
- ~100 lignes
|
||||
|
||||
2. **IT_002_FileSystemWrite** : `tests/integration/IT_002_FileSystemWrite.cpp`
|
||||
- Appelle tool `filesystem_write` → créer `test_output.md`
|
||||
- Vérifie fichier créé avec bon contenu
|
||||
- ~120 lignes
|
||||
|
||||
3. **IT_003_FileSystemRead** : `tests/integration/IT_003_FileSystemRead.cpp`
|
||||
- Appelle tool `filesystem_read` sur fichier existant
|
||||
- Vérifie contenu retourné correct
|
||||
- ~120 lignes
|
||||
|
||||
4. **IT_004_MCPToolsList** : `tests/integration/IT_004_MCPToolsList.cpp`
|
||||
- Requête AI : "Liste tous tes tools disponibles"
|
||||
- Vérifie que LLM retourne les 17+ tools
|
||||
- Parse JSON et compte les tools
|
||||
- ~150 lignes
|
||||
|
||||
#### Tests Flux Complets (Priorité HAUTE)
|
||||
|
||||
5. **IT_005_VoiceToAI** : `tests/integration/IT_005_VoiceToAI.cpp`
|
||||
- Simule transcription voice → `voice:transcription`
|
||||
- Vérifie AI reçoit et publie `llm:request`
|
||||
- ~120 lignes
|
||||
|
||||
6. **IT_006_AIToLLM** : `tests/integration/IT_006_AIToLLM.cpp`
|
||||
- Publie `ai:query` avec vraie question
|
||||
- Vérifie `llm:response` reçue de Claude API
|
||||
- Vérifie réponse cohérente (non vide, pas d'erreur)
|
||||
- ~150 lignes
|
||||
|
||||
7. **IT_007_StorageWrite** : `tests/integration/IT_007_StorageWrite.cpp`
|
||||
- Demande AI d'écrire note via tool `storage_save_note`
|
||||
- Vérifie fichier .md créé dans `data/notes/`
|
||||
- ~130 lignes
|
||||
|
||||
8. **IT_008_StorageRead** : `tests/integration/IT_008_StorageRead.cpp`
|
||||
- Demande AI de lire note via tool `storage_query_notes`
|
||||
- Vérifie contenu retourné correct
|
||||
- ~130 lignes
|
||||
|
||||
9. **IT_009_FullConversationLoop** : `tests/integration/IT_009_FullConversationLoop.cpp`
|
||||
- Flux complet : Voice → AI → LLM (écrit note) → Storage → LLM (lit note) → Voice
|
||||
- Scénario : "Prends note que j'aime le C++" → "Qu'est-ce que j'aime ?"
|
||||
- Vérifie chaque étape du flux
|
||||
- ~250 lignes
|
||||
|
||||
#### Tests Modules de Base (Priorité MOYENNE)
|
||||
|
||||
10. **IT_010_SchedulerHyperfocus** : `tests/integration/IT_010_SchedulerHyperfocus.cpp`
|
||||
- Simule session longue (>120min)
|
||||
- Vérifie `scheduler:hyperfocus_detected` publié
|
||||
- ~100 lignes
|
||||
|
||||
11. **IT_011_NotificationAlert** : `tests/integration/IT_011_NotificationAlert.cpp`
|
||||
- Publie `notification:alert`
|
||||
- Vérifie message affiché (check logs)
|
||||
- ~100 lignes
|
||||
|
||||
12. **IT_012_MonitoringActivity** : `tests/integration/IT_012_MonitoringActivity.cpp`
|
||||
- Simule `platform:window_changed`
|
||||
- Vérifie Monitoring track correctement
|
||||
- ~100 lignes
|
||||
|
||||
13. **IT_013_WebRequest** : `tests/integration/IT_013_WebRequest.cpp`
|
||||
- Publie `web:request` vers https://api.github.com
|
||||
- Vérifie `web:response` avec statusCode 200
|
||||
- ~100 lignes
|
||||
|
||||
---
|
||||
|
||||
## 4. Protocole de Test
|
||||
|
||||
### 4.1 Structure d'un Test Module
|
||||
|
||||
Exemple : `IT_001_GetCurrentTime.cpp`
|
||||
|
||||
```cpp
|
||||
#include "shared/testing/ITestModule.h"
|
||||
#include <grove/IIO.h>
|
||||
#include <grove/JsonDataNode.h>
|
||||
#include <chrono>
|
||||
|
||||
namespace aissia::testing {
|
||||
|
||||
class IT_001_GetCurrentTime : public ITestModule {
|
||||
public:
|
||||
std::string getTestName() const override {
|
||||
return "IT_001_GetCurrentTime";
|
||||
}
|
||||
|
||||
std::string getDescription() const override {
|
||||
return "Test MCP tool get_current_time via AI";
|
||||
}
|
||||
|
||||
void setConfiguration(const grove::IDataNode& config,
|
||||
grove::IIO* io,
|
||||
grove::ITaskScheduler* scheduler) override {
|
||||
m_io = io;
|
||||
m_timeout = config.getInt("timeoutMs", 10000);
|
||||
|
||||
// Subscribe to responses
|
||||
grove::SubscriptionConfig subConfig;
|
||||
m_io->subscribe("llm:response", subConfig);
|
||||
}
|
||||
|
||||
TestResult execute() override {
|
||||
auto start = std::chrono::steady_clock::now();
|
||||
TestResult result;
|
||||
result.testName = getTestName();
|
||||
|
||||
try {
|
||||
// 1. Envoyer requête AI pour appeler tool
|
||||
auto request = std::make_unique<grove::JsonDataNode>("request");
|
||||
request->setString("query", "Quelle heure est-il ?");
|
||||
request->setString("conversationId", "test-001");
|
||||
m_io->publish("ai:query", std::move(request));
|
||||
|
||||
// 2. Attendre réponse (avec timeout)
|
||||
auto response = waitForResponse("llm:response", m_timeout);
|
||||
|
||||
if (!response) {
|
||||
result.passed = false;
|
||||
result.message = "Timeout waiting for llm:response";
|
||||
return result;
|
||||
}
|
||||
|
||||
// 3. Valider réponse contient timestamp
|
||||
std::string text = response->getString("text", "");
|
||||
if (text.empty()) {
|
||||
result.passed = false;
|
||||
result.message = "Empty response from LLM";
|
||||
return result;
|
||||
}
|
||||
|
||||
// 4. Parse pour vérifier format heure (simple regex)
|
||||
bool hasTimestamp = (text.find(":") != std::string::npos) &&
|
||||
(text.find("20") != std::string::npos); // Year 2025
|
||||
|
||||
result.passed = hasTimestamp;
|
||||
result.message = hasTimestamp ? "✅ Tool returned valid time"
|
||||
: "❌ No valid timestamp in response";
|
||||
result.details["response"] = text;
|
||||
|
||||
} catch (const std::exception& e) {
|
||||
result.passed = false;
|
||||
result.message = std::string("Exception: ") + e.what();
|
||||
}
|
||||
|
||||
auto end = std::chrono::steady_clock::now();
|
||||
result.durationMs = std::chrono::duration_cast<std::chrono::milliseconds>(
|
||||
end - start).count();
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
private:
|
||||
std::unique_ptr<grove::IDataNode> waitForResponse(
|
||||
const std::string& topic, int timeoutMs) {
|
||||
// Polling avec timeout
|
||||
auto start = std::chrono::steady_clock::now();
|
||||
while (true) {
|
||||
if (m_io->hasMessages() > 0) {
|
||||
auto msg = m_io->pullMessage();
|
||||
if (msg.topic == topic && msg.data) {
|
||||
return std::move(msg.data);
|
||||
}
|
||||
}
|
||||
|
||||
auto elapsed = std::chrono::duration_cast<std::chrono::milliseconds>(
|
||||
std::chrono::steady_clock::now() - start).count();
|
||||
if (elapsed > timeoutMs) {
|
||||
return nullptr;
|
||||
}
|
||||
|
||||
std::this_thread::sleep_for(std::chrono::milliseconds(100));
|
||||
}
|
||||
}
|
||||
|
||||
grove::IIO* m_io = nullptr;
|
||||
int m_timeout = 10000;
|
||||
};
|
||||
|
||||
} // namespace aissia::testing
|
||||
|
||||
// Factory functions
|
||||
extern "C" {
|
||||
grove::IModule* createModule() {
|
||||
return new aissia::testing::IT_001_GetCurrentTime();
|
||||
}
|
||||
|
||||
void destroyModule(grove::IModule* module) {
|
||||
delete module;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 4.2 Configuration par Test
|
||||
|
||||
Chaque test a son JSON : `config/integration/IT_001.json`
|
||||
|
||||
```json
|
||||
{
|
||||
"enabled": true,
|
||||
"timeoutMs": 10000,
|
||||
"retryCount": 0,
|
||||
"description": "Test MCP tool get_current_time",
|
||||
"tags": ["mcp", "tools", "quick"]
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. Format des Résultats
|
||||
|
||||
### 5.1 Console Output
|
||||
|
||||
```
|
||||
========================================
|
||||
AISSIA Integration Tests
|
||||
Running 13 tests...
|
||||
========================================
|
||||
|
||||
[1/13] IT_001_GetCurrentTime..................... ✅ PASS (1.2s)
|
||||
Tool returned valid time
|
||||
|
||||
[2/13] IT_002_FileSystemWrite.................... ✅ PASS (0.8s)
|
||||
File created: data/test_output.md
|
||||
|
||||
[3/13] IT_003_FileSystemRead..................... ✅ PASS (0.5s)
|
||||
Content matches expected
|
||||
|
||||
[4/13] IT_004_MCPToolsList....................... ✅ PASS (2.3s)
|
||||
Found 17 tools available
|
||||
|
||||
[5/13] IT_005_VoiceToAI.......................... ✅ PASS (0.3s)
|
||||
AI received transcription
|
||||
|
||||
[6/13] IT_006_AIToLLM............................ ✅ PASS (3.5s)
|
||||
LLM response received (234 tokens)
|
||||
|
||||
[7/13] IT_007_StorageWrite....................... ✅ PASS (1.1s)
|
||||
Note saved to data/notes/test-note.md
|
||||
|
||||
[8/13] IT_008_StorageRead........................ ✅ PASS (0.9s)
|
||||
Note retrieved successfully
|
||||
|
||||
[9/13] IT_009_FullConversationLoop............... ✅ PASS (8.7s)
|
||||
Complete loop: Voice→AI→LLM→Storage→Voice
|
||||
|
||||
[10/13] IT_010_SchedulerHyperfocus............... ✅ PASS (0.2s)
|
||||
Hyperfocus detected correctly
|
||||
|
||||
[11/13] IT_011_NotificationAlert................. ✅ PASS (0.1s)
|
||||
Alert published
|
||||
|
||||
[12/13] IT_012_MonitoringActivity................ ❌ FAIL (5.0s)
|
||||
Timeout waiting for monitoring:activity_classified
|
||||
|
||||
[13/13] IT_013_WebRequest........................ ✅ PASS (0.6s)
|
||||
HTTP 200 from api.github.com
|
||||
|
||||
========================================
|
||||
Results: 12/13 passed (92.3%)
|
||||
Total time: 25.2s
|
||||
Failed tests:
|
||||
- IT_012_MonitoringActivity: Timeout waiting for monitoring:activity_classified
|
||||
========================================
|
||||
|
||||
Exit code: 1
|
||||
```
|
||||
|
||||
### 5.2 JSON Output
|
||||
|
||||
`test-results.json` :
|
||||
|
||||
```json
|
||||
{
|
||||
"summary": {
|
||||
"total": 13,
|
||||
"passed": 12,
|
||||
"failed": 1,
|
||||
"skipped": 0,
|
||||
"successRate": 92.3,
|
||||
"totalDurationMs": 25200
|
||||
},
|
||||
"tests": [
|
||||
{
|
||||
"name": "IT_001_GetCurrentTime",
|
||||
"passed": true,
|
||||
"message": "Tool returned valid time",
|
||||
"durationMs": 1200,
|
||||
"details": {
|
||||
"response": "Il est actuellement 17:45:23 le 28 novembre 2025."
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "IT_012_MonitoringActivity",
|
||||
"passed": false,
|
||||
"message": "Timeout waiting for monitoring:activity_classified",
|
||||
"durationMs": 5000,
|
||||
"details": {
|
||||
"expectedTopic": "monitoring:activity_classified",
|
||||
"timeout": 5000
|
||||
}
|
||||
}
|
||||
],
|
||||
"timestamp": "2025-11-28T17:45:30Z",
|
||||
"environment": {
|
||||
"platform": "linux",
|
||||
"modules": ["Scheduler", "Notification", "Monitoring", "AI", "Voice", "Storage", "Web"],
|
||||
"llmProvider": "claude-sonnet-4"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 6. Implémentation par Phases
|
||||
|
||||
### Phase 1 : Infrastructure (2h)
|
||||
|
||||
**Objectif** : Créer le système de base
|
||||
|
||||
1. **ITestModule interface** (`src/shared/testing/ITestModule.h`)
|
||||
- Définir interface commune
|
||||
- Structure TestResult
|
||||
|
||||
2. **TestRunnerModule** (`src/modules/TestRunnerModule.{h,cpp}`)
|
||||
- Découverte de tests
|
||||
- Chargement dynamique
|
||||
- Collecte résultats
|
||||
- Génération rapport
|
||||
|
||||
3. **Configuration**
|
||||
- `config/test_runner.json`
|
||||
- `config/integration/` (dossier pour configs de tests)
|
||||
|
||||
4. **Intégration main.cpp**
|
||||
- Argument `--run-tests`
|
||||
- Mode test vs mode normal
|
||||
|
||||
### Phase 2 : Tests MCP (3h)
|
||||
|
||||
**Objectif** : Valider les tools MCP
|
||||
|
||||
5. **IT_001_GetCurrentTime** - Test simple de tool
|
||||
6. **IT_002_FileSystemWrite** - Écriture fichier .md
|
||||
7. **IT_003_FileSystemRead** - Lecture fichier .md
|
||||
8. **IT_004_MCPToolsList** - Inventaire complet des tools
|
||||
|
||||
### Phase 3 : Tests Flux (3h)
|
||||
|
||||
**Objectif** : Valider les communications inter-modules
|
||||
|
||||
9. **IT_005_VoiceToAI** - Voice → AI
|
||||
10. **IT_006_AIToLLM** - AI → LLM (Claude API réelle)
|
||||
11. **IT_007_StorageWrite** - AI → Storage (sauvegarder note)
|
||||
12. **IT_008_StorageRead** - AI → Storage (lire note)
|
||||
|
||||
### Phase 4 : Test Complet (2h)
|
||||
|
||||
**Objectif** : Valider le flux end-to-end
|
||||
|
||||
13. **IT_009_FullConversationLoop** - Boucle complète Voice→AI→LLM→Storage→LLM→Voice
|
||||
|
||||
### Phase 5 : Tests Modules (1h)
|
||||
|
||||
**Objectif** : Valider modules individuels
|
||||
|
||||
14. **IT_010_SchedulerHyperfocus**
|
||||
15. **IT_011_NotificationAlert**
|
||||
16. **IT_012_MonitoringActivity**
|
||||
17. **IT_013_WebRequest**
|
||||
|
||||
### Phase 6 : Finition (1h)
|
||||
|
||||
18. Documentation
|
||||
19. Validation complète
|
||||
20. Git commit
|
||||
|
||||
**Total estimé : ~12h**
|
||||
|
||||
---
|
||||
|
||||
## 7. CMakeLists.txt
|
||||
|
||||
```cmake
|
||||
# ============================================================================
|
||||
# Integration Test Modules
|
||||
# ============================================================================
|
||||
|
||||
# Test Runner Module (orchestrator)
|
||||
add_library(TestRunnerModule SHARED
|
||||
src/modules/TestRunnerModule.cpp
|
||||
)
|
||||
target_link_libraries(TestRunnerModule PRIVATE
|
||||
GroveEngine::impl
|
||||
spdlog::spdlog
|
||||
)
|
||||
set_target_properties(TestRunnerModule PROPERTIES
|
||||
PREFIX "lib"
|
||||
LIBRARY_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/modules
|
||||
)
|
||||
|
||||
# Individual test modules
|
||||
set(INTEGRATION_TESTS
|
||||
IT_001_GetCurrentTime
|
||||
IT_002_FileSystemWrite
|
||||
IT_003_FileSystemRead
|
||||
IT_004_MCPToolsList
|
||||
IT_005_VoiceToAI
|
||||
IT_006_AIToLLM
|
||||
IT_007_StorageWrite
|
||||
IT_008_StorageRead
|
||||
IT_009_FullConversationLoop
|
||||
IT_010_SchedulerHyperfocus
|
||||
IT_011_NotificationAlert
|
||||
IT_012_MonitoringActivity
|
||||
IT_013_WebRequest
|
||||
)
|
||||
|
||||
foreach(TEST_NAME ${INTEGRATION_TESTS})
|
||||
add_library(${TEST_NAME} SHARED
|
||||
tests/integration/${TEST_NAME}.cpp
|
||||
)
|
||||
target_include_directories(${TEST_NAME} PRIVATE
|
||||
${CMAKE_CURRENT_SOURCE_DIR}/src
|
||||
)
|
||||
target_link_libraries(${TEST_NAME} PRIVATE
|
||||
GroveEngine::impl
|
||||
spdlog::spdlog
|
||||
)
|
||||
set_target_properties(${TEST_NAME} PROPERTIES
|
||||
PREFIX ""
|
||||
LIBRARY_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/tests/integration
|
||||
)
|
||||
endforeach()
|
||||
|
||||
# Custom target to build all integration tests
|
||||
add_custom_target(integration_tests
|
||||
DEPENDS TestRunnerModule ${INTEGRATION_TESTS}
|
||||
COMMENT "Building all integration test modules"
|
||||
)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 8. Utilisation
|
||||
|
||||
### 8.1 Build
|
||||
|
||||
```bash
|
||||
# Build tous les tests
|
||||
cmake --build build --target integration_tests -j4
|
||||
|
||||
# Vérifier modules créés
|
||||
ls build/tests/integration/
|
||||
# IT_001_GetCurrentTime.so
|
||||
# IT_002_FileSystemWrite.so
|
||||
# ...
|
||||
```
|
||||
|
||||
### 8.2 Exécution
|
||||
|
||||
```bash
|
||||
# Lancer tous les tests
|
||||
./build/aissia --run-tests
|
||||
|
||||
# Avec config custom
|
||||
./build/aissia --run-tests --test-config config/my_tests.json
|
||||
|
||||
# Verbose mode
|
||||
./build/aissia --run-tests --verbose
|
||||
|
||||
# Sauvegarder résultats JSON
|
||||
./build/aissia --run-tests --json-output results.json
|
||||
```
|
||||
|
||||
### 8.3 CI/CD
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
# ci-test.sh
|
||||
|
||||
set -e
|
||||
|
||||
# Build
|
||||
cmake -B build -DBUILD_TESTING=ON
|
||||
cmake --build build --target integration_tests -j4
|
||||
|
||||
# Run tests
|
||||
./build/aissia --run-tests --json-output test-results.json
|
||||
|
||||
# Exit code: 0 = success, 1 = failure
|
||||
if [ $? -eq 0 ]; then
|
||||
echo "✅ All tests passed!"
|
||||
exit 0
|
||||
else
|
||||
echo "❌ Some tests failed"
|
||||
cat test-results.json
|
||||
exit 1
|
||||
fi
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 9. Critères de Succès
|
||||
|
||||
### 9.1 Tests Unitaires (Catch2)
|
||||
|
||||
✅ **120/120 tests passent** (déjà fait)
|
||||
|
||||
### 9.2 Tests d'Intégration
|
||||
|
||||
✅ **13/13 tests passent** en conditions réelles :
|
||||
- MCP tools fonctionnent
|
||||
- LLM Claude API répond
|
||||
- Storage écrit/lit fichiers .md
|
||||
- Flux complets Voice→AI→LLM→Storage→Voice
|
||||
|
||||
### 9.3 Performance
|
||||
|
||||
✅ **Suite complète < 60s** (temps total)
|
||||
✅ **Chaque test < 10s** (sauf FullConversationLoop < 15s)
|
||||
|
||||
### 9.4 Fiabilité
|
||||
|
||||
✅ **Tests reproductibles** (pas de flakiness)
|
||||
✅ **Isolation** : Un test qui fail ne bloque pas les autres
|
||||
✅ **Cleanup** : Fichiers .md de test nettoyés après exécution
|
||||
|
||||
---
|
||||
|
||||
## 10. Extensions Futures (Optionnel)
|
||||
|
||||
### Phase 7+ : Features Avancées
|
||||
|
||||
- **IT_014_HotReload** : Test du hot-reload pendant exécution
|
||||
- **IT_015_ConcurrentRequests** : Test charge (multiple AI queries)
|
||||
- **IT_016_ErrorRecovery** : Test résilience (LLM down → fallback)
|
||||
- **IT_017_MCPExternalServer** : Test MCP server externe
|
||||
- **IT_018_MultimodalInput** : Test image + texte
|
||||
|
||||
### Monitoring
|
||||
|
||||
- Dashboard web pour visualiser résultats
|
||||
- Historique des runs (trend analysis)
|
||||
- Alertes si taux de succès < 90%
|
||||
|
||||
---
|
||||
|
||||
## 11. Checklist de Validation
|
||||
|
||||
Avant de considérer le travail terminé :
|
||||
|
||||
- [ ] TestRunnerModule compile et charge
|
||||
- [ ] Au moins 3 tests MCP passent
|
||||
- [ ] Au moins 1 flux complet passe (IT_009)
|
||||
- [ ] Rapport console clair et lisible
|
||||
- [ ] JSON output valide et parsable
|
||||
- [ ] Exit code correct (0/1)
|
||||
- [ ] Documentation à jour
|
||||
- [ ] Commit avec message clair
|
||||
|
||||
---
|
||||
|
||||
## 12. Fichiers Affectés
|
||||
|
||||
### Nouveaux fichiers
|
||||
|
||||
```
|
||||
src/shared/testing/ITestModule.h
|
||||
src/modules/TestRunnerModule.h
|
||||
src/modules/TestRunnerModule.cpp
|
||||
config/test_runner.json
|
||||
config/integration/IT_001.json
|
||||
config/integration/IT_002.json
|
||||
...
|
||||
tests/integration/IT_001_GetCurrentTime.cpp
|
||||
tests/integration/IT_002_FileSystemWrite.cpp
|
||||
tests/integration/IT_003_FileSystemRead.cpp
|
||||
tests/integration/IT_004_MCPToolsList.cpp
|
||||
tests/integration/IT_005_VoiceToAI.cpp
|
||||
tests/integration/IT_006_AIToLLM.cpp
|
||||
tests/integration/IT_007_StorageWrite.cpp
|
||||
tests/integration/IT_008_StorageRead.cpp
|
||||
tests/integration/IT_009_FullConversationLoop.cpp
|
||||
tests/integration/IT_010_SchedulerHyperfocus.cpp
|
||||
tests/integration/IT_011_NotificationAlert.cpp
|
||||
tests/integration/IT_012_MonitoringActivity.cpp
|
||||
tests/integration/IT_013_WebRequest.cpp
|
||||
plans/integration-tests-plan.md (ce fichier)
|
||||
```
|
||||
|
||||
### Fichiers modifiés
|
||||
|
||||
```
|
||||
CMakeLists.txt # Ajouter integration_tests target
|
||||
src/main.cpp # Ajouter --run-tests flag
|
||||
README.md # Documenter système de tests
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
**Plan prêt pour implémentation** 🚀
|
||||
|
||||
Ce plan détaille un système de tests d'intégration innovant qui utilise l'architecture modulaire de GroveEngine pour rendre chaque test isolé, extensible et hot-reloadable. L'approche "un module = un test" démontre la puissance du système tout en fournissant une validation complète d'AISSIA en conditions réelles.
|
||||
45
src/main.cpp
45
src/main.cpp
@ -264,6 +264,16 @@ int main(int argc, char* argv[]) {
|
||||
return runMCPServer();
|
||||
}
|
||||
|
||||
// Check for test runner mode
|
||||
bool testMode = false;
|
||||
for (int i = 1; i < argc; ++i) {
|
||||
if (std::strcmp(argv[i], "--run-tests") == 0 ||
|
||||
std::strcmp(argv[i], "--test") == 0) {
|
||||
testMode = true;
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
// Check for interactive mode
|
||||
for (int i = 1; i < argc; ++i) {
|
||||
if (std::strcmp(argv[i], "--interactive") == 0 ||
|
||||
@ -279,10 +289,15 @@ int main(int argc, char* argv[]) {
|
||||
spdlog::set_pattern("[%H:%M:%S.%e] [%n] [%^%l%$] %v");
|
||||
|
||||
spdlog::info("========================================");
|
||||
spdlog::info(" AISSIA - Assistant Personnel IA");
|
||||
if (testMode) {
|
||||
spdlog::info(" AISSIA - Integration Test Mode");
|
||||
} else {
|
||||
spdlog::info(" AISSIA - Assistant Personnel IA");
|
||||
}
|
||||
spdlog::info(" Powered by GroveEngine");
|
||||
spdlog::info(" Architecture: Services + Hot-Reload Modules");
|
||||
spdlog::info(" (Use --mcp-server to run as MCP server)");
|
||||
spdlog::info(" (Use --run-tests to run integration tests)");
|
||||
spdlog::info("========================================");
|
||||
|
||||
// Signal handling
|
||||
@ -429,15 +444,25 @@ int main(int argc, char* argv[]) {
|
||||
}
|
||||
|
||||
// Liste des modules a charger (sans infrastructure)
|
||||
std::vector<std::pair<std::string, std::string>> moduleList = {
|
||||
{"SchedulerModule", "scheduler.json"},
|
||||
{"NotificationModule", "notification.json"},
|
||||
{"MonitoringModule", "monitoring.json"},
|
||||
{"AIModule", "ai.json"},
|
||||
{"VoiceModule", "voice.json"},
|
||||
{"StorageModule", "storage.json"},
|
||||
{"WebModule", "web.json"},
|
||||
};
|
||||
std::vector<std::pair<std::string, std::string>> moduleList;
|
||||
|
||||
if (testMode) {
|
||||
// In test mode, only load TestRunnerModule
|
||||
moduleList = {
|
||||
{"TestRunnerModule", "test_runner.json"}
|
||||
};
|
||||
} else {
|
||||
// Normal mode: load all regular modules
|
||||
moduleList = {
|
||||
{"SchedulerModule", "scheduler.json"},
|
||||
{"NotificationModule", "notification.json"},
|
||||
{"MonitoringModule", "monitoring.json"},
|
||||
{"AIModule", "ai.json"},
|
||||
{"VoiceModule", "voice.json"},
|
||||
{"StorageModule", "storage.json"},
|
||||
{"WebModule", "web.json"},
|
||||
};
|
||||
}
|
||||
|
||||
// Charger les modules
|
||||
for (const auto& [moduleName, configFile] : moduleList) {
|
||||
|
||||
336
src/modules/TestRunnerModule.cpp
Normal file
336
src/modules/TestRunnerModule.cpp
Normal file
@ -0,0 +1,336 @@
|
||||
#include "TestRunnerModule.h"
|
||||
#include <grove/JsonDataNode.h>
|
||||
#include <spdlog/spdlog.h>
|
||||
#include <filesystem>
|
||||
#include <fstream>
|
||||
#include <chrono>
|
||||
#include <dlfcn.h>
|
||||
|
||||
namespace fs = std::filesystem;
|
||||
|
||||
namespace aissia {
|
||||
|
||||
TestRunnerModule::TestRunnerModule() = default;
|
||||
TestRunnerModule::~TestRunnerModule() = default;
|
||||
|
||||
void TestRunnerModule::setConfiguration(const grove::IDataNode& config,
|
||||
grove::IIO* io,
|
||||
grove::ITaskScheduler* scheduler) {
|
||||
m_io = io;
|
||||
m_scheduler = scheduler;
|
||||
m_config = std::make_unique<grove::JsonDataNode>("config");
|
||||
|
||||
m_testDirectory = config.getString("testDirectory", "tests/integration");
|
||||
m_globalTimeoutMs = config.getInt("globalTimeoutMs", 300000);
|
||||
m_stopOnFirstFailure = config.getBool("stopOnFirstFailure", false);
|
||||
m_verboseOutput = config.getBool("verboseOutput", true);
|
||||
m_jsonOutputPath = config.getString("jsonOutputPath", "test-results.json");
|
||||
|
||||
spdlog::info("[TestRunner] Configuration loaded:");
|
||||
spdlog::info(" Test directory: {}", m_testDirectory);
|
||||
spdlog::info(" Global timeout: {}ms", m_globalTimeoutMs);
|
||||
spdlog::info(" Stop on first failure: {}", m_stopOnFirstFailure);
|
||||
|
||||
discoverTests();
|
||||
}
|
||||
|
||||
const grove::IDataNode& TestRunnerModule::getConfiguration() {
|
||||
return *m_config;
|
||||
}
|
||||
|
||||
void TestRunnerModule::discoverTests() {
|
||||
m_testPaths.clear();
|
||||
|
||||
fs::path testDir("build/" + m_testDirectory);
|
||||
if (!fs::exists(testDir)) {
|
||||
spdlog::warn("[TestRunner] Test directory not found: {}", testDir.string());
|
||||
return;
|
||||
}
|
||||
|
||||
for (const auto& entry : fs::directory_iterator(testDir)) {
|
||||
if (entry.is_regular_file()) {
|
||||
std::string filename = entry.path().filename().string();
|
||||
if (filename.find("IT_") == 0 && entry.path().extension() == ".so") {
|
||||
m_testPaths.push_back(entry.path().string());
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
std::sort(m_testPaths.begin(), m_testPaths.end());
|
||||
|
||||
spdlog::info("[TestRunner] Discovered {} test(s)", m_testPaths.size());
|
||||
for (const auto& path : m_testPaths) {
|
||||
spdlog::info(" - {}", fs::path(path).filename().string());
|
||||
}
|
||||
}
|
||||
|
||||
testing::TestResult TestRunnerModule::runTest(const std::string& testPath) {
|
||||
testing::TestResult result;
|
||||
auto startTime = std::chrono::steady_clock::now();
|
||||
|
||||
// Load the test module
|
||||
void* handle = dlopen(testPath.c_str(), RTLD_LAZY);
|
||||
if (!handle) {
|
||||
result.passed = false;
|
||||
result.testName = fs::path(testPath).stem().string();
|
||||
result.message = std::string("Failed to load module: ") + dlerror();
|
||||
spdlog::error("[TestRunner] {}", result.message);
|
||||
return result;
|
||||
}
|
||||
|
||||
// Get createModule function
|
||||
using CreateModuleFn = grove::IModule* (*)();
|
||||
auto createModule = reinterpret_cast<CreateModuleFn>(dlsym(handle, "createModule"));
|
||||
if (!createModule) {
|
||||
result.passed = false;
|
||||
result.testName = fs::path(testPath).stem().string();
|
||||
result.message = "createModule symbol not found";
|
||||
dlclose(handle);
|
||||
return result;
|
||||
}
|
||||
|
||||
// Create test instance
|
||||
auto* module = createModule();
|
||||
auto* testModule = dynamic_cast<testing::ITestModule*>(module);
|
||||
if (!testModule) {
|
||||
result.passed = false;
|
||||
result.testName = fs::path(testPath).stem().string();
|
||||
result.message = "Module does not implement ITestModule";
|
||||
delete module;
|
||||
dlclose(handle);
|
||||
return result;
|
||||
}
|
||||
|
||||
// Configure test module
|
||||
grove::JsonDataNode config("test_config");
|
||||
config.setInt("timeoutMs", 10000);
|
||||
testModule->setConfiguration(config, m_io, m_scheduler);
|
||||
|
||||
// Execute test
|
||||
try {
|
||||
result = testModule->execute();
|
||||
} catch (const std::exception& e) {
|
||||
result.passed = false;
|
||||
result.testName = testModule->getTestName();
|
||||
result.message = std::string("Exception: ") + e.what();
|
||||
}
|
||||
|
||||
auto endTime = std::chrono::steady_clock::now();
|
||||
result.durationMs = std::chrono::duration_cast<std::chrono::milliseconds>(
|
||||
endTime - startTime).count();
|
||||
|
||||
// Cleanup
|
||||
using DestroyModuleFn = void (*)(grove::IModule*);
|
||||
auto destroyModule = reinterpret_cast<DestroyModuleFn>(dlsym(handle, "destroyModule"));
|
||||
if (destroyModule) {
|
||||
destroyModule(module);
|
||||
} else {
|
||||
delete module;
|
||||
}
|
||||
dlclose(handle);
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
void TestRunnerModule::process(const grove::IDataNode& input) {
|
||||
if (m_executed) {
|
||||
return; // Tests already run
|
||||
}
|
||||
|
||||
m_executed = true;
|
||||
|
||||
spdlog::info("========================================");
|
||||
spdlog::info(" AISSIA Integration Tests");
|
||||
spdlog::info(" Running {} test(s)...", m_testPaths.size());
|
||||
spdlog::info("========================================");
|
||||
|
||||
auto globalStart = std::chrono::steady_clock::now();
|
||||
|
||||
for (size_t i = 0; i < m_testPaths.size(); ++i) {
|
||||
const auto& testPath = m_testPaths[i];
|
||||
std::string testName = fs::path(testPath).stem().string();
|
||||
|
||||
if (m_verboseOutput) {
|
||||
spdlog::info("[{}/{}] {}...", i + 1, m_testPaths.size(), testName);
|
||||
}
|
||||
|
||||
auto result = runTest(testPath);
|
||||
m_results.push_back(result);
|
||||
|
||||
std::string status = result.passed ? "✅ PASS" : "❌ FAIL";
|
||||
spdlog::info("[{}/{}] {}... {} ({:.1f}s)",
|
||||
i + 1, m_testPaths.size(), testName, status,
|
||||
result.durationMs / 1000.0);
|
||||
|
||||
if (m_verboseOutput && !result.message.empty()) {
|
||||
spdlog::info(" {}", result.message);
|
||||
}
|
||||
|
||||
if (!result.passed && m_stopOnFirstFailure) {
|
||||
spdlog::warn("[TestRunner] Stopping on first failure");
|
||||
break;
|
||||
}
|
||||
|
||||
// Check global timeout
|
||||
auto elapsed = std::chrono::duration_cast<std::chrono::milliseconds>(
|
||||
std::chrono::steady_clock::now() - globalStart).count();
|
||||
if (elapsed > m_globalTimeoutMs) {
|
||||
spdlog::error("[TestRunner] Global timeout exceeded ({}ms)", m_globalTimeoutMs);
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
generateReport();
|
||||
|
||||
if (!m_jsonOutputPath.empty()) {
|
||||
generateJsonReport(m_jsonOutputPath);
|
||||
}
|
||||
|
||||
// Determine exit code
|
||||
int failedCount = 0;
|
||||
for (const auto& result : m_results) {
|
||||
if (!result.passed) {
|
||||
failedCount++;
|
||||
}
|
||||
}
|
||||
|
||||
int exitCode = failedCount == 0 ? 0 : 1;
|
||||
spdlog::info("Exit code: {}", exitCode);
|
||||
|
||||
// Exit the application
|
||||
std::exit(exitCode);
|
||||
}
|
||||
|
||||
void TestRunnerModule::generateReport() {
|
||||
spdlog::info("========================================");
|
||||
|
||||
int passed = 0;
|
||||
int failed = 0;
|
||||
int totalDuration = 0;
|
||||
|
||||
for (const auto& result : m_results) {
|
||||
if (result.passed) {
|
||||
passed++;
|
||||
} else {
|
||||
failed++;
|
||||
}
|
||||
totalDuration += result.durationMs;
|
||||
}
|
||||
|
||||
int total = passed + failed;
|
||||
double successRate = total > 0 ? (100.0 * passed) / total : 0.0;
|
||||
|
||||
spdlog::info("Results: {}/{} passed ({:.1f}%)", passed, total, successRate);
|
||||
spdlog::info("Total time: {:.1f}s", totalDuration / 1000.0);
|
||||
|
||||
if (failed > 0) {
|
||||
spdlog::info("Failed tests:");
|
||||
for (const auto& result : m_results) {
|
||||
if (!result.passed) {
|
||||
spdlog::info(" - {}: {}", result.testName, result.message);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
spdlog::info("========================================");
|
||||
}
|
||||
|
||||
void TestRunnerModule::generateJsonReport(const std::string& outputPath) {
|
||||
nlohmann::json report;
|
||||
|
||||
// Summary
|
||||
int passed = 0;
|
||||
int failed = 0;
|
||||
int totalDuration = 0;
|
||||
|
||||
for (const auto& result : m_results) {
|
||||
if (result.passed) {
|
||||
passed++;
|
||||
} else {
|
||||
failed++;
|
||||
}
|
||||
totalDuration += result.durationMs;
|
||||
}
|
||||
|
||||
int total = passed + failed;
|
||||
double successRate = total > 0 ? (100.0 * passed) / total : 0.0;
|
||||
|
||||
report["summary"] = {
|
||||
{"total", total},
|
||||
{"passed", passed},
|
||||
{"failed", failed},
|
||||
{"skipped", 0},
|
||||
{"successRate", successRate},
|
||||
{"totalDurationMs", totalDuration}
|
||||
};
|
||||
|
||||
// Individual tests
|
||||
nlohmann::json tests = nlohmann::json::array();
|
||||
for (const auto& result : m_results) {
|
||||
nlohmann::json testJson = {
|
||||
{"name", result.testName},
|
||||
{"passed", result.passed},
|
||||
{"message", result.message},
|
||||
{"durationMs", result.durationMs},
|
||||
{"details", result.details}
|
||||
};
|
||||
tests.push_back(testJson);
|
||||
}
|
||||
report["tests"] = tests;
|
||||
|
||||
// Metadata
|
||||
auto now = std::chrono::system_clock::now();
|
||||
auto timestamp = std::chrono::system_clock::to_time_t(now);
|
||||
char buf[100];
|
||||
std::strftime(buf, sizeof(buf), "%Y-%m-%dT%H:%M:%SZ", std::gmtime(×tamp));
|
||||
|
||||
report["timestamp"] = buf;
|
||||
report["environment"] = {
|
||||
{"platform", "linux"},
|
||||
{"testDirectory", m_testDirectory}
|
||||
};
|
||||
|
||||
// Write to file
|
||||
std::ofstream file(outputPath);
|
||||
if (file.is_open()) {
|
||||
file << report.dump(2);
|
||||
file.close();
|
||||
spdlog::info("[TestRunner] JSON report written to: {}", outputPath);
|
||||
} else {
|
||||
spdlog::error("[TestRunner] Failed to write JSON report to: {}", outputPath);
|
||||
}
|
||||
}
|
||||
|
||||
std::unique_ptr<grove::IDataNode> TestRunnerModule::getHealthStatus() {
|
||||
auto status = std::make_unique<grove::JsonDataNode>("health");
|
||||
status->setString("status", "healthy");
|
||||
status->setInt("testsRun", m_results.size());
|
||||
return status;
|
||||
}
|
||||
|
||||
void TestRunnerModule::shutdown() {
|
||||
spdlog::info("[TestRunner] Shutdown");
|
||||
}
|
||||
|
||||
std::unique_ptr<grove::IDataNode> TestRunnerModule::getState() {
|
||||
auto state = std::make_unique<grove::JsonDataNode>("state");
|
||||
state->setBool("executed", m_executed);
|
||||
return state;
|
||||
}
|
||||
|
||||
void TestRunnerModule::setState(const grove::IDataNode& state) {
|
||||
m_executed = state.getBool("executed", false);
|
||||
}
|
||||
|
||||
} // namespace aissia
|
||||
|
||||
// Factory functions
|
||||
extern "C" {
|
||||
grove::IModule* createModule() {
|
||||
return new aissia::TestRunnerModule();
|
||||
}
|
||||
|
||||
void destroyModule(grove::IModule* module) {
|
||||
delete module;
|
||||
}
|
||||
}
|
||||
79
src/modules/TestRunnerModule.h
Normal file
79
src/modules/TestRunnerModule.h
Normal file
@ -0,0 +1,79 @@
|
||||
#pragma once
|
||||
|
||||
#include <grove/IModule.h>
|
||||
#include <grove/IIO.h>
|
||||
#include <grove/ITaskScheduler.h>
|
||||
#include <grove/JsonDataNode.h>
|
||||
#include <shared/testing/ITestModule.h>
|
||||
#include <string>
|
||||
#include <vector>
|
||||
#include <memory>
|
||||
|
||||
namespace aissia {
|
||||
|
||||
/**
|
||||
* @brief Orchestrator for integration tests
|
||||
*
|
||||
* TestRunnerModule discovers, loads, and executes integration test modules.
|
||||
* Each test is a separate .so file that implements ITestModule.
|
||||
*
|
||||
* Workflow:
|
||||
* 1. Scan tests/integration/ for IT_*.so files
|
||||
* 2. For each test:
|
||||
* - Load module dynamically
|
||||
* - Execute via execute()
|
||||
* - Collect result
|
||||
* - Unload module
|
||||
* 3. Generate report (console + JSON)
|
||||
* 4. Exit with appropriate code (0 = success, 1 = failure)
|
||||
*/
|
||||
class TestRunnerModule : public grove::IModule {
|
||||
public:
|
||||
TestRunnerModule();
|
||||
~TestRunnerModule() override;
|
||||
|
||||
void setConfiguration(const grove::IDataNode& config,
|
||||
grove::IIO* io,
|
||||
grove::ITaskScheduler* scheduler) override;
|
||||
|
||||
void process(const grove::IDataNode& input) override;
|
||||
|
||||
const grove::IDataNode& getConfiguration() override;
|
||||
|
||||
std::unique_ptr<grove::IDataNode> getHealthStatus() override;
|
||||
|
||||
void shutdown() override;
|
||||
|
||||
std::unique_ptr<grove::IDataNode> getState() override;
|
||||
|
||||
void setState(const grove::IDataNode& state) override;
|
||||
|
||||
std::string getType() const override { return "TestRunnerModule"; }
|
||||
|
||||
int getVersion() const override { return 1; }
|
||||
|
||||
bool isIdle() const override { return m_executed; }
|
||||
|
||||
private:
|
||||
void discoverTests();
|
||||
testing::TestResult runTest(const std::string& testPath);
|
||||
void generateReport();
|
||||
void generateJsonReport(const std::string& outputPath);
|
||||
|
||||
grove::IIO* m_io = nullptr;
|
||||
grove::ITaskScheduler* m_scheduler = nullptr;
|
||||
std::unique_ptr<grove::JsonDataNode> m_config;
|
||||
|
||||
std::string m_testDirectory;
|
||||
int m_globalTimeoutMs = 300000; // 5 minutes
|
||||
bool m_stopOnFirstFailure = false;
|
||||
bool m_verboseOutput = true;
|
||||
std::string m_jsonOutputPath;
|
||||
|
||||
std::vector<std::string> m_testPaths;
|
||||
std::vector<testing::TestResult> m_results;
|
||||
|
||||
bool m_executed = false;
|
||||
};
|
||||
|
||||
} // namespace aissia
|
||||
48
src/shared/testing/ITestModule.h
Normal file
48
src/shared/testing/ITestModule.h
Normal file
@ -0,0 +1,48 @@
|
||||
#pragma once
|
||||
|
||||
#include <grove/IModule.h>
|
||||
#include <string>
|
||||
#include <nlohmann/json.hpp>
|
||||
|
||||
namespace aissia::testing {
|
||||
|
||||
/**
|
||||
* @brief Result of a test execution
|
||||
*/
|
||||
struct TestResult {
|
||||
bool passed = false;
|
||||
std::string testName;
|
||||
std::string message;
|
||||
int durationMs = 0;
|
||||
nlohmann::json details; // Additional test-specific data
|
||||
};
|
||||
|
||||
/**
|
||||
* @brief Base interface for integration test modules
|
||||
*
|
||||
* All integration tests inherit from this interface.
|
||||
* Each test is a dynamically loaded module that executes
|
||||
* a specific scenario and returns a result.
|
||||
*/
|
||||
class ITestModule : public grove::IModule {
|
||||
public:
|
||||
virtual ~ITestModule() = default;
|
||||
|
||||
/**
|
||||
* @brief Execute the test
|
||||
* @return TestResult with pass/fail status and details
|
||||
*/
|
||||
virtual TestResult execute() = 0;
|
||||
|
||||
/**
|
||||
* @brief Get unique test name (e.g., "IT_001_GetCurrentTime")
|
||||
*/
|
||||
virtual std::string getTestName() const = 0;
|
||||
|
||||
/**
|
||||
* @brief Get human-readable test description
|
||||
*/
|
||||
virtual std::string getDescription() const = 0;
|
||||
};
|
||||
|
||||
} // namespace aissia::testing
|
||||
@ -106,3 +106,62 @@ add_custom_target(test_mcp
|
||||
DEPENDS aissia_tests
|
||||
COMMENT "Running MCP integration tests"
|
||||
)
|
||||
|
||||
# ============================================================================
|
||||
# Integration Test Modules (Dynamic .so files)
|
||||
# ============================================================================
|
||||
|
||||
# Helper macro to create integration test modules
|
||||
macro(add_integration_test TEST_NAME)
|
||||
add_library(${TEST_NAME} SHARED
|
||||
integration/${TEST_NAME}.cpp
|
||||
)
|
||||
target_include_directories(${TEST_NAME} PRIVATE
|
||||
${CMAKE_SOURCE_DIR}/src
|
||||
)
|
||||
target_link_libraries(${TEST_NAME} PRIVATE
|
||||
GroveEngine::impl
|
||||
spdlog::spdlog
|
||||
)
|
||||
set_target_properties(${TEST_NAME} PROPERTIES
|
||||
PREFIX ""
|
||||
LIBRARY_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/tests/integration
|
||||
)
|
||||
endmacro()
|
||||
|
||||
# Individual integration test modules (will be added as we create them)
|
||||
# Phase 2: MCP Tests
|
||||
add_integration_test(IT_001_GetCurrentTime)
|
||||
add_integration_test(IT_002_FileSystemWrite)
|
||||
add_integration_test(IT_003_FileSystemRead)
|
||||
add_integration_test(IT_004_MCPToolsList)
|
||||
|
||||
# Phase 3: Flow Tests
|
||||
add_integration_test(IT_005_VoiceToAI)
|
||||
add_integration_test(IT_006_AIToLLM)
|
||||
add_integration_test(IT_007_StorageWrite)
|
||||
add_integration_test(IT_008_StorageRead)
|
||||
|
||||
# Phase 4: End-to-End Test
|
||||
# add_integration_test(IT_009_FullConversationLoop)
|
||||
|
||||
# Phase 5: Module Tests
|
||||
# add_integration_test(IT_010_SchedulerHyperfocus)
|
||||
# add_integration_test(IT_011_NotificationAlert)
|
||||
# add_integration_test(IT_012_MonitoringActivity)
|
||||
# add_integration_test(IT_013_WebRequest)
|
||||
|
||||
# Custom target to build all integration tests
|
||||
add_custom_target(integration_tests
|
||||
DEPENDS
|
||||
IT_001_GetCurrentTime
|
||||
IT_002_FileSystemWrite
|
||||
IT_003_FileSystemRead
|
||||
IT_004_MCPToolsList
|
||||
IT_005_VoiceToAI
|
||||
IT_006_AIToLLM
|
||||
IT_007_StorageWrite
|
||||
IT_008_StorageRead
|
||||
COMMENT "Building all integration test modules"
|
||||
)
|
||||
|
||||
|
||||
176
tests/integration/IT_001_GetCurrentTime.cpp
Normal file
176
tests/integration/IT_001_GetCurrentTime.cpp
Normal file
@ -0,0 +1,176 @@
|
||||
#include <shared/testing/ITestModule.h>
|
||||
#include <grove/JsonDataNode.h>
|
||||
#include <grove/IIO.h>
|
||||
#include <spdlog/spdlog.h>
|
||||
#include <chrono>
|
||||
#include <thread>
|
||||
|
||||
namespace aissia::testing {
|
||||
|
||||
/**
|
||||
* @brief Test MCP tool get_current_time via AIModule
|
||||
*
|
||||
* Workflow:
|
||||
* 1. Publish ai:query with "Quelle heure est-il ?"
|
||||
* 2. Wait for llm:response (timeout 30s)
|
||||
* 3. Validate response contains timestamp
|
||||
*/
|
||||
class IT_001_GetCurrentTime : public ITestModule {
|
||||
public:
|
||||
std::string getTestName() const override {
|
||||
return "IT_001_GetCurrentTime";
|
||||
}
|
||||
|
||||
std::string getDescription() const override {
|
||||
return "Test MCP tool get_current_time via AI";
|
||||
}
|
||||
|
||||
void setConfiguration(const grove::IDataNode& config,
|
||||
grove::IIO* io,
|
||||
grove::ITaskScheduler* scheduler) override {
|
||||
m_io = io;
|
||||
m_scheduler = scheduler;
|
||||
m_timeout = config.getInt("timeoutMs", 30000); // 30s for LLM
|
||||
|
||||
// Subscribe to LLM response
|
||||
grove::SubscriptionConfig subConfig;
|
||||
m_io->subscribe("llm:response", subConfig);
|
||||
m_io->subscribe("llm:error", subConfig);
|
||||
|
||||
spdlog::info("[{}] Configured with timeout={}ms", getTestName(), m_timeout);
|
||||
}
|
||||
|
||||
void process(const grove::IDataNode& input) override {
|
||||
// Not used in test mode
|
||||
}
|
||||
|
||||
void shutdown() override {}
|
||||
|
||||
const grove::IDataNode& getConfiguration() override {
|
||||
static grove::JsonDataNode config("config");
|
||||
return config;
|
||||
}
|
||||
|
||||
std::unique_ptr<grove::IDataNode> getHealthStatus() override {
|
||||
auto status = std::make_unique<grove::JsonDataNode>("health");
|
||||
status->setString("status", "healthy");
|
||||
return status;
|
||||
}
|
||||
|
||||
std::unique_ptr<grove::IDataNode> getState() override {
|
||||
return std::make_unique<grove::JsonDataNode>("state");
|
||||
}
|
||||
|
||||
void setState(const grove::IDataNode& state) override {}
|
||||
|
||||
std::string getType() const override { return "IT_001_GetCurrentTime"; }
|
||||
int getVersion() const override { return 1; }
|
||||
bool isIdle() const override { return true; }
|
||||
|
||||
TestResult execute() override {
|
||||
auto start = std::chrono::steady_clock::now();
|
||||
TestResult result;
|
||||
result.testName = getTestName();
|
||||
|
||||
try {
|
||||
spdlog::info("[{}] Sending query to AI...", getTestName());
|
||||
|
||||
// 1. Send query to AI
|
||||
auto request = std::make_unique<grove::JsonDataNode>("request");
|
||||
request->setString("query", "Quelle heure est-il exactement maintenant ?");
|
||||
request->setString("conversationId", "it001");
|
||||
m_io->publish("ai:query", std::move(request));
|
||||
|
||||
// 2. Wait for response
|
||||
auto response = waitForMessage("llm:response", m_timeout);
|
||||
|
||||
if (!response) {
|
||||
// Check for error message
|
||||
auto error = waitForMessage("llm:error", 1000);
|
||||
if (error) {
|
||||
result.passed = false;
|
||||
result.message = "LLM error: " + error->getString("message", "Unknown");
|
||||
result.details["error"] = error->getString("message", "");
|
||||
} else {
|
||||
result.passed = false;
|
||||
result.message = "Timeout waiting for llm:response";
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
// 3. Validate response
|
||||
std::string text = response->getString("text", "");
|
||||
if (text.empty()) {
|
||||
result.passed = false;
|
||||
result.message = "Empty response from LLM";
|
||||
return result;
|
||||
}
|
||||
|
||||
spdlog::info("[{}] Received response: {}", getTestName(), text);
|
||||
|
||||
// 4. Check for time indicators (simple heuristic)
|
||||
bool hasTime = (text.find(":") != std::string::npos) &&
|
||||
(text.find("h") != std::string::npos ||
|
||||
text.find("H") != std::string::npos ||
|
||||
std::isdigit(text[0]));
|
||||
|
||||
result.passed = hasTime;
|
||||
result.message = hasTime ? "Tool returned valid time"
|
||||
: "No valid timestamp in response";
|
||||
result.details["response"] = text;
|
||||
|
||||
} catch (const std::exception& e) {
|
||||
result.passed = false;
|
||||
result.message = std::string("Exception: ") + e.what();
|
||||
spdlog::error("[{}] {}", getTestName(), result.message);
|
||||
}
|
||||
|
||||
auto end = std::chrono::steady_clock::now();
|
||||
result.durationMs = std::chrono::duration_cast<std::chrono::milliseconds>(
|
||||
end - start).count();
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
private:
|
||||
std::unique_ptr<grove::IDataNode> waitForMessage(
|
||||
const std::string& topic, int timeoutMs) {
|
||||
|
||||
auto start = std::chrono::steady_clock::now();
|
||||
|
||||
while (true) {
|
||||
if (m_io->hasMessages() > 0) {
|
||||
auto msg = m_io->pullMessage();
|
||||
if (msg.topic == topic && msg.data) {
|
||||
return std::move(msg.data);
|
||||
}
|
||||
}
|
||||
|
||||
auto elapsed = std::chrono::duration_cast<std::chrono::milliseconds>(
|
||||
std::chrono::steady_clock::now() - start).count();
|
||||
|
||||
if (elapsed > timeoutMs) {
|
||||
return nullptr;
|
||||
}
|
||||
|
||||
std::this_thread::sleep_for(std::chrono::milliseconds(100));
|
||||
}
|
||||
}
|
||||
|
||||
grove::IIO* m_io = nullptr;
|
||||
grove::ITaskScheduler* m_scheduler = nullptr;
|
||||
int m_timeout = 30000;
|
||||
};
|
||||
|
||||
} // namespace aissia::testing
|
||||
|
||||
// Factory functions
|
||||
extern "C" {
|
||||
grove::IModule* createModule() {
|
||||
return new aissia::testing::IT_001_GetCurrentTime();
|
||||
}
|
||||
|
||||
void destroyModule(grove::IModule* module) {
|
||||
delete module;
|
||||
}
|
||||
}
|
||||
198
tests/integration/IT_002_FileSystemWrite.cpp
Normal file
198
tests/integration/IT_002_FileSystemWrite.cpp
Normal file
@ -0,0 +1,198 @@
|
||||
#include <shared/testing/ITestModule.h>
|
||||
#include <grove/JsonDataNode.h>
|
||||
#include <grove/IIO.h>
|
||||
#include <spdlog/spdlog.h>
|
||||
#include <chrono>
|
||||
#include <thread>
|
||||
#include <filesystem>
|
||||
#include <fstream>
|
||||
|
||||
namespace fs = std::filesystem;
|
||||
|
||||
namespace aissia::testing {
|
||||
|
||||
/**
|
||||
* @brief Test MCP tool filesystem_write via AIModule
|
||||
*
|
||||
* Workflow:
|
||||
* 1. Publish ai:query requesting file write
|
||||
* 2. Wait for llm:response (timeout 30s)
|
||||
* 3. Verify file was created with correct content
|
||||
*/
|
||||
class IT_002_FileSystemWrite : public ITestModule {
|
||||
public:
|
||||
std::string getTestName() const override {
|
||||
return "IT_002_FileSystemWrite";
|
||||
}
|
||||
|
||||
std::string getDescription() const override {
|
||||
return "Test MCP tool filesystem_write";
|
||||
}
|
||||
|
||||
void setConfiguration(const grove::IDataNode& config,
|
||||
grove::IIO* io,
|
||||
grove::ITaskScheduler* scheduler) override {
|
||||
m_io = io;
|
||||
m_scheduler = scheduler;
|
||||
m_timeout = config.getInt("timeoutMs", 30000);
|
||||
|
||||
grove::SubscriptionConfig subConfig;
|
||||
m_io->subscribe("llm:response", subConfig);
|
||||
m_io->subscribe("llm:error", subConfig);
|
||||
|
||||
spdlog::info("[{}] Configured", getTestName());
|
||||
}
|
||||
|
||||
void process(const grove::IDataNode& input) override {}
|
||||
void shutdown() override {}
|
||||
|
||||
const grove::IDataNode& getConfiguration() override {
|
||||
static grove::JsonDataNode config("config");
|
||||
return config;
|
||||
}
|
||||
|
||||
std::unique_ptr<grove::IDataNode> getHealthStatus() override {
|
||||
auto status = std::make_unique<grove::JsonDataNode>("health");
|
||||
status->setString("status", "healthy");
|
||||
return status;
|
||||
}
|
||||
|
||||
std::unique_ptr<grove::IDataNode> getState() override {
|
||||
return std::make_unique<grove::JsonDataNode>("state");
|
||||
}
|
||||
|
||||
void setState(const grove::IDataNode& state) override {}
|
||||
|
||||
std::string getType() const override { return "IT_002_FileSystemWrite"; }
|
||||
int getVersion() const override { return 1; }
|
||||
bool isIdle() const override { return true; }
|
||||
|
||||
TestResult execute() override {
|
||||
auto start = std::chrono::steady_clock::now();
|
||||
TestResult result;
|
||||
result.testName = getTestName();
|
||||
|
||||
const std::string testFile = "data/test_output_it002.md";
|
||||
const std::string testContent = "Test IT_002 réussi - Integration test write";
|
||||
|
||||
try {
|
||||
// Cleanup previous test file if exists
|
||||
if (fs::exists(testFile)) {
|
||||
fs::remove(testFile);
|
||||
spdlog::info("[{}] Cleaned up previous test file", getTestName());
|
||||
}
|
||||
|
||||
spdlog::info("[{}] Requesting file write via AI...", getTestName());
|
||||
|
||||
// 1. Send query to AI
|
||||
auto request = std::make_unique<grove::JsonDataNode>("request");
|
||||
request->setString("query",
|
||||
"Utilise le tool filesystem_write pour écrire le texte '" +
|
||||
testContent + "' dans le fichier '" + testFile + "'");
|
||||
request->setString("conversationId", "it002");
|
||||
m_io->publish("ai:query", std::move(request));
|
||||
|
||||
// 2. Wait for response
|
||||
auto response = waitForMessage("llm:response", m_timeout);
|
||||
|
||||
if (!response) {
|
||||
auto error = waitForMessage("llm:error", 1000);
|
||||
if (error) {
|
||||
result.passed = false;
|
||||
result.message = "LLM error: " + error->getString("message", "Unknown");
|
||||
} else {
|
||||
result.passed = false;
|
||||
result.message = "Timeout waiting for llm:response";
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
std::string text = response->getString("text", "");
|
||||
spdlog::info("[{}] LLM response: {}", getTestName(), text);
|
||||
|
||||
// 3. Give LLM time to execute tool (async)
|
||||
std::this_thread::sleep_for(std::chrono::milliseconds(2000));
|
||||
|
||||
// 4. Verify file was created
|
||||
if (!fs::exists(testFile)) {
|
||||
result.passed = false;
|
||||
result.message = "File was not created: " + testFile;
|
||||
result.details["response"] = text;
|
||||
return result;
|
||||
}
|
||||
|
||||
// 5. Verify content
|
||||
std::ifstream file(testFile);
|
||||
std::string content((std::istreambuf_iterator<char>(file)),
|
||||
std::istreambuf_iterator<char>());
|
||||
file.close();
|
||||
|
||||
bool contentMatches = content.find(testContent) != std::string::npos;
|
||||
|
||||
result.passed = contentMatches;
|
||||
result.message = contentMatches ?
|
||||
"File created with correct content" :
|
||||
"File created but content mismatch";
|
||||
result.details["filePath"] = testFile;
|
||||
result.details["expectedContent"] = testContent;
|
||||
result.details["actualContent"] = content;
|
||||
|
||||
// Cleanup
|
||||
if (fs::exists(testFile)) {
|
||||
fs::remove(testFile);
|
||||
}
|
||||
|
||||
} catch (const std::exception& e) {
|
||||
result.passed = false;
|
||||
result.message = std::string("Exception: ") + e.what();
|
||||
spdlog::error("[{}] {}", getTestName(), result.message);
|
||||
}
|
||||
|
||||
auto end = std::chrono::steady_clock::now();
|
||||
result.durationMs = std::chrono::duration_cast<std::chrono::milliseconds>(
|
||||
end - start).count();
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
private:
|
||||
std::unique_ptr<grove::IDataNode> waitForMessage(
|
||||
const std::string& topic, int timeoutMs) {
|
||||
|
||||
auto start = std::chrono::steady_clock::now();
|
||||
|
||||
while (true) {
|
||||
if (m_io->hasMessages() > 0) {
|
||||
auto msg = m_io->pullMessage();
|
||||
if (msg.topic == topic && msg.data) {
|
||||
return std::move(msg.data);
|
||||
}
|
||||
}
|
||||
|
||||
auto elapsed = std::chrono::duration_cast<std::chrono::milliseconds>(
|
||||
std::chrono::steady_clock::now() - start).count();
|
||||
|
||||
if (elapsed > timeoutMs) {
|
||||
return nullptr;
|
||||
}
|
||||
|
||||
std::this_thread::sleep_for(std::chrono::milliseconds(100));
|
||||
}
|
||||
}
|
||||
|
||||
grove::IIO* m_io = nullptr;
|
||||
grove::ITaskScheduler* m_scheduler = nullptr;
|
||||
int m_timeout = 30000;
|
||||
};
|
||||
|
||||
} // namespace aissia::testing
|
||||
|
||||
extern "C" {
|
||||
grove::IModule* createModule() {
|
||||
return new aissia::testing::IT_002_FileSystemWrite();
|
||||
}
|
||||
|
||||
void destroyModule(grove::IModule* module) {
|
||||
delete module;
|
||||
}
|
||||
}
|
||||
188
tests/integration/IT_003_FileSystemRead.cpp
Normal file
188
tests/integration/IT_003_FileSystemRead.cpp
Normal file
@ -0,0 +1,188 @@
|
||||
#include <shared/testing/ITestModule.h>
|
||||
#include <grove/JsonDataNode.h>
|
||||
#include <grove/IIO.h>
|
||||
#include <spdlog/spdlog.h>
|
||||
#include <chrono>
|
||||
#include <thread>
|
||||
#include <filesystem>
|
||||
#include <fstream>
|
||||
|
||||
namespace fs = std::filesystem;
|
||||
|
||||
namespace aissia::testing {
|
||||
|
||||
/**
|
||||
* @brief Test MCP tool filesystem_read via AIModule
|
||||
*
|
||||
* Workflow:
|
||||
* 1. Create test file with known content
|
||||
* 2. Publish ai:query requesting file read
|
||||
* 3. Wait for llm:response
|
||||
* 4. Verify response contains file content
|
||||
*/
|
||||
class IT_003_FileSystemRead : public ITestModule {
|
||||
public:
|
||||
std::string getTestName() const override {
|
||||
return "IT_003_FileSystemRead";
|
||||
}
|
||||
|
||||
std::string getDescription() const override {
|
||||
return "Test MCP tool filesystem_read";
|
||||
}
|
||||
|
||||
void setConfiguration(const grove::IDataNode& config,
|
||||
grove::IIO* io,
|
||||
grove::ITaskScheduler* scheduler) override {
|
||||
m_io = io;
|
||||
m_scheduler = scheduler;
|
||||
m_timeout = config.getInt("timeoutMs", 30000);
|
||||
|
||||
grove::SubscriptionConfig subConfig;
|
||||
m_io->subscribe("llm:response", subConfig);
|
||||
m_io->subscribe("llm:error", subConfig);
|
||||
|
||||
spdlog::info("[{}] Configured", getTestName());
|
||||
}
|
||||
|
||||
void process(const grove::IDataNode& input) override {}
|
||||
void shutdown() override {}
|
||||
|
||||
const grove::IDataNode& getConfiguration() override {
|
||||
static grove::JsonDataNode config("config");
|
||||
return config;
|
||||
}
|
||||
|
||||
std::unique_ptr<grove::IDataNode> getHealthStatus() override {
|
||||
auto status = std::make_unique<grove::JsonDataNode>("health");
|
||||
status->setString("status", "healthy");
|
||||
return status;
|
||||
}
|
||||
|
||||
std::unique_ptr<grove::IDataNode> getState() override {
|
||||
return std::make_unique<grove::JsonDataNode>("state");
|
||||
}
|
||||
|
||||
void setState(const grove::IDataNode& state) override {}
|
||||
|
||||
std::string getType() const override { return "IT_003_FileSystemRead"; }
|
||||
int getVersion() const override { return 1; }
|
||||
bool isIdle() const override { return true; }
|
||||
|
||||
TestResult execute() override {
|
||||
auto start = std::chrono::steady_clock::now();
|
||||
TestResult result;
|
||||
result.testName = getTestName();
|
||||
|
||||
const std::string testFile = "data/test_input_it003.md";
|
||||
const std::string testContent = "Hello World from IT_003 integration test";
|
||||
|
||||
try {
|
||||
// 1. Create test file
|
||||
fs::create_directories("data");
|
||||
std::ofstream file(testFile);
|
||||
file << testContent;
|
||||
file.close();
|
||||
spdlog::info("[{}] Created test file: {}", getTestName(), testFile);
|
||||
|
||||
// 2. Send query to AI
|
||||
spdlog::info("[{}] Requesting file read via AI...", getTestName());
|
||||
auto request = std::make_unique<grove::JsonDataNode>("request");
|
||||
request->setString("query",
|
||||
"Utilise le tool filesystem_read pour lire le contenu du fichier '" +
|
||||
testFile + "' et dis-moi ce qu'il contient");
|
||||
request->setString("conversationId", "it003");
|
||||
m_io->publish("ai:query", std::move(request));
|
||||
|
||||
// 3. Wait for response
|
||||
auto response = waitForMessage("llm:response", m_timeout);
|
||||
|
||||
if (!response) {
|
||||
auto error = waitForMessage("llm:error", 1000);
|
||||
if (error) {
|
||||
result.passed = false;
|
||||
result.message = "LLM error: " + error->getString("message", "Unknown");
|
||||
} else {
|
||||
result.passed = false;
|
||||
result.message = "Timeout waiting for llm:response";
|
||||
}
|
||||
// Cleanup
|
||||
if (fs::exists(testFile)) fs::remove(testFile);
|
||||
return result;
|
||||
}
|
||||
|
||||
std::string text = response->getString("text", "");
|
||||
spdlog::info("[{}] LLM response: {}", getTestName(), text);
|
||||
|
||||
// 4. Verify response contains the content
|
||||
bool contentFound = text.find("Hello World") != std::string::npos ||
|
||||
text.find("IT_003") != std::string::npos;
|
||||
|
||||
result.passed = contentFound;
|
||||
result.message = contentFound ?
|
||||
"Content read correctly" :
|
||||
"Expected content not found in response";
|
||||
result.details["expectedContent"] = testContent;
|
||||
result.details["response"] = text;
|
||||
|
||||
// Cleanup
|
||||
if (fs::exists(testFile)) {
|
||||
fs::remove(testFile);
|
||||
}
|
||||
|
||||
} catch (const std::exception& e) {
|
||||
result.passed = false;
|
||||
result.message = std::string("Exception: ") + e.what();
|
||||
spdlog::error("[{}] {}", getTestName(), result.message);
|
||||
|
||||
// Cleanup on error
|
||||
if (fs::exists(testFile)) fs::remove(testFile);
|
||||
}
|
||||
|
||||
auto end = std::chrono::steady_clock::now();
|
||||
result.durationMs = std::chrono::duration_cast<std::chrono::milliseconds>(
|
||||
end - start).count();
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
private:
|
||||
std::unique_ptr<grove::IDataNode> waitForMessage(
|
||||
const std::string& topic, int timeoutMs) {
|
||||
|
||||
auto start = std::chrono::steady_clock::now();
|
||||
|
||||
while (true) {
|
||||
if (m_io->hasMessages() > 0) {
|
||||
auto msg = m_io->pullMessage();
|
||||
if (msg.topic == topic && msg.data) {
|
||||
return std::move(msg.data);
|
||||
}
|
||||
}
|
||||
|
||||
auto elapsed = std::chrono::duration_cast<std::chrono::milliseconds>(
|
||||
std::chrono::steady_clock::now() - start).count();
|
||||
|
||||
if (elapsed > timeoutMs) {
|
||||
return nullptr;
|
||||
}
|
||||
|
||||
std::this_thread::sleep_for(std::chrono::milliseconds(100));
|
||||
}
|
||||
}
|
||||
|
||||
grove::IIO* m_io = nullptr;
|
||||
grove::ITaskScheduler* m_scheduler = nullptr;
|
||||
int m_timeout = 30000;
|
||||
};
|
||||
|
||||
} // namespace aissia::testing
|
||||
|
||||
extern "C" {
|
||||
grove::IModule* createModule() {
|
||||
return new aissia::testing::IT_003_FileSystemRead();
|
||||
}
|
||||
|
||||
void destroyModule(grove::IModule* module) {
|
||||
delete module;
|
||||
}
|
||||
}
|
||||
189
tests/integration/IT_004_MCPToolsList.cpp
Normal file
189
tests/integration/IT_004_MCPToolsList.cpp
Normal file
@ -0,0 +1,189 @@
|
||||
#include <shared/testing/ITestModule.h>
|
||||
#include <grove/JsonDataNode.h>
|
||||
#include <grove/IIO.h>
|
||||
#include <spdlog/spdlog.h>
|
||||
#include <chrono>
|
||||
#include <thread>
|
||||
|
||||
namespace aissia::testing {
|
||||
|
||||
/**
|
||||
* @brief Test MCP tools inventory
|
||||
*
|
||||
* Workflow:
|
||||
* 1. Publish ai:query requesting list of available tools
|
||||
* 2. Wait for llm:response
|
||||
* 3. Verify response mentions multiple tools (at least 5)
|
||||
*/
|
||||
class IT_004_MCPToolsList : public ITestModule {
|
||||
public:
|
||||
std::string getTestName() const override {
|
||||
return "IT_004_MCPToolsList";
|
||||
}
|
||||
|
||||
std::string getDescription() const override {
|
||||
return "Test MCP tools inventory";
|
||||
}
|
||||
|
||||
void setConfiguration(const grove::IDataNode& config,
|
||||
grove::IIO* io,
|
||||
grove::ITaskScheduler* scheduler) override {
|
||||
m_io = io;
|
||||
m_scheduler = scheduler;
|
||||
m_timeout = config.getInt("timeoutMs", 30000);
|
||||
|
||||
grove::SubscriptionConfig subConfig;
|
||||
m_io->subscribe("llm:response", subConfig);
|
||||
m_io->subscribe("llm:error", subConfig);
|
||||
|
||||
spdlog::info("[{}] Configured", getTestName());
|
||||
}
|
||||
|
||||
void process(const grove::IDataNode& input) override {}
|
||||
void shutdown() override {}
|
||||
|
||||
const grove::IDataNode& getConfiguration() override {
|
||||
static grove::JsonDataNode config("config");
|
||||
return config;
|
||||
}
|
||||
|
||||
std::unique_ptr<grove::IDataNode> getHealthStatus() override {
|
||||
auto status = std::make_unique<grove::JsonDataNode>("health");
|
||||
status->setString("status", "healthy");
|
||||
return status;
|
||||
}
|
||||
|
||||
std::unique_ptr<grove::IDataNode> getState() override {
|
||||
return std::make_unique<grove::JsonDataNode>("state");
|
||||
}
|
||||
|
||||
void setState(const grove::IDataNode& state) override {}
|
||||
|
||||
std::string getType() const override { return "IT_004_MCPToolsList"; }
|
||||
int getVersion() const override { return 1; }
|
||||
bool isIdle() const override { return true; }
|
||||
|
||||
TestResult execute() override {
|
||||
auto start = std::chrono::steady_clock::now();
|
||||
TestResult result;
|
||||
result.testName = getTestName();
|
||||
|
||||
try {
|
||||
spdlog::info("[{}] Requesting tools list from AI...", getTestName());
|
||||
|
||||
// 1. Send query to AI
|
||||
auto request = std::make_unique<grove::JsonDataNode>("request");
|
||||
request->setString("query",
|
||||
"Liste tous les tools (outils) dont tu disposes. "
|
||||
"Pour chaque tool, donne son nom et sa description.");
|
||||
request->setString("conversationId", "it004");
|
||||
m_io->publish("ai:query", std::move(request));
|
||||
|
||||
// 2. Wait for response
|
||||
auto response = waitForMessage("llm:response", m_timeout);
|
||||
|
||||
if (!response) {
|
||||
auto error = waitForMessage("llm:error", 1000);
|
||||
if (error) {
|
||||
result.passed = false;
|
||||
result.message = "LLM error: " + error->getString("message", "Unknown");
|
||||
} else {
|
||||
result.passed = false;
|
||||
result.message = "Timeout waiting for llm:response";
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
std::string text = response->getString("text", "");
|
||||
spdlog::info("[{}] LLM response (length={}): {}",
|
||||
getTestName(), text.length(), text.substr(0, 200));
|
||||
|
||||
// 3. Count mentions of expected tools
|
||||
std::vector<std::string> expectedTools = {
|
||||
"get_current_time",
|
||||
"filesystem_read",
|
||||
"filesystem_write",
|
||||
"filesystem_list",
|
||||
"storage_save_note",
|
||||
"storage_query_notes",
|
||||
"storage_get_note",
|
||||
"storage_delete_note",
|
||||
"storage_update_note"
|
||||
};
|
||||
|
||||
int toolsFound = 0;
|
||||
std::string foundTools;
|
||||
|
||||
for (const auto& toolName : expectedTools) {
|
||||
if (text.find(toolName) != std::string::npos) {
|
||||
toolsFound++;
|
||||
if (!foundTools.empty()) foundTools += ", ";
|
||||
foundTools += toolName;
|
||||
}
|
||||
}
|
||||
|
||||
// Accept if at least 5 tools are mentioned (flexible for LLM response variations)
|
||||
bool passed = toolsFound >= 5;
|
||||
|
||||
result.passed = passed;
|
||||
result.message = passed ?
|
||||
"Found " + std::to_string(toolsFound) + " tools in response" :
|
||||
"Only found " + std::to_string(toolsFound) + " tools (expected >= 5)";
|
||||
result.details["toolsFound"] = toolsFound;
|
||||
result.details["toolsList"] = foundTools;
|
||||
result.details["response"] = text.substr(0, 500); // First 500 chars
|
||||
|
||||
} catch (const std::exception& e) {
|
||||
result.passed = false;
|
||||
result.message = std::string("Exception: ") + e.what();
|
||||
spdlog::error("[{}] {}", getTestName(), result.message);
|
||||
}
|
||||
|
||||
auto end = std::chrono::steady_clock::now();
|
||||
result.durationMs = std::chrono::duration_cast<std::chrono::milliseconds>(
|
||||
end - start).count();
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
private:
|
||||
std::unique_ptr<grove::IDataNode> waitForMessage(
|
||||
const std::string& topic, int timeoutMs) {
|
||||
|
||||
auto start = std::chrono::steady_clock::now();
|
||||
|
||||
while (true) {
|
||||
if (m_io->hasMessages() > 0) {
|
||||
auto msg = m_io->pullMessage();
|
||||
if (msg.topic == topic && msg.data) {
|
||||
return std::move(msg.data);
|
||||
}
|
||||
}
|
||||
|
||||
auto elapsed = std::chrono::duration_cast<std::chrono::milliseconds>(
|
||||
std::chrono::steady_clock::now() - start).count();
|
||||
|
||||
if (elapsed > timeoutMs) {
|
||||
return nullptr;
|
||||
}
|
||||
|
||||
std::this_thread::sleep_for(std::chrono::milliseconds(100));
|
||||
}
|
||||
}
|
||||
|
||||
grove::IIO* m_io = nullptr;
|
||||
grove::ITaskScheduler* m_scheduler = nullptr;
|
||||
int m_timeout = 30000;
|
||||
};
|
||||
|
||||
} // namespace aissia::testing
|
||||
|
||||
extern "C" {
|
||||
grove::IModule* createModule() {
|
||||
return new aissia::testing::IT_004_MCPToolsList();
|
||||
}
|
||||
|
||||
void destroyModule(grove::IModule* module) {
|
||||
delete module;
|
||||
}
|
||||
}
|
||||
154
tests/integration/IT_005_VoiceToAI.cpp
Normal file
154
tests/integration/IT_005_VoiceToAI.cpp
Normal file
@ -0,0 +1,154 @@
|
||||
#include <shared/testing/ITestModule.h>
|
||||
#include <grove/JsonDataNode.h>
|
||||
#include <grove/IIO.h>
|
||||
#include <spdlog/spdlog.h>
|
||||
#include <chrono>
|
||||
#include <thread>
|
||||
|
||||
namespace aissia::testing {
|
||||
|
||||
/**
|
||||
* @brief Test Voice → AI communication flow
|
||||
*
|
||||
* Workflow:
|
||||
* 1. Publish voice:transcription (simulating voice input)
|
||||
* 2. Wait for ai:query or llm:request (AIModule processing)
|
||||
* 3. Verify AIModule received and forwarded the transcription
|
||||
*/
|
||||
class IT_005_VoiceToAI : public ITestModule {
|
||||
public:
|
||||
std::string getTestName() const override {
|
||||
return "IT_005_VoiceToAI";
|
||||
}
|
||||
|
||||
std::string getDescription() const override {
|
||||
return "Test Voice → AI communication flow";
|
||||
}
|
||||
|
||||
void setConfiguration(const grove::IDataNode& config,
|
||||
grove::IIO* io,
|
||||
grove::ITaskScheduler* scheduler) override {
|
||||
m_io = io;
|
||||
m_scheduler = scheduler;
|
||||
m_timeout = config.getInt("timeoutMs", 10000);
|
||||
|
||||
// Subscribe to LLM request to detect AI processing
|
||||
grove::SubscriptionConfig subConfig;
|
||||
m_io->subscribe("llm:request", subConfig);
|
||||
m_io->subscribe("llm:response", subConfig);
|
||||
|
||||
spdlog::info("[{}] Configured", getTestName());
|
||||
}
|
||||
|
||||
void process(const grove::IDataNode& input) override {}
|
||||
void shutdown() override {}
|
||||
|
||||
const grove::IDataNode& getConfiguration() override {
|
||||
static grove::JsonDataNode config("config");
|
||||
return config;
|
||||
}
|
||||
|
||||
std::unique_ptr<grove::IDataNode> getHealthStatus() override {
|
||||
auto status = std::make_unique<grove::JsonDataNode>("health");
|
||||
status->setString("status", "healthy");
|
||||
return status;
|
||||
}
|
||||
|
||||
std::unique_ptr<grove::IDataNode> getState() override {
|
||||
return std::make_unique<grove::JsonDataNode>("state");
|
||||
}
|
||||
|
||||
void setState(const grove::IDataNode& state) override {}
|
||||
|
||||
std::string getType() const override { return "IT_005_VoiceToAI"; }
|
||||
int getVersion() const override { return 1; }
|
||||
bool isIdle() const override { return true; }
|
||||
|
||||
TestResult execute() override {
|
||||
auto start = std::chrono::steady_clock::now();
|
||||
TestResult result;
|
||||
result.testName = getTestName();
|
||||
|
||||
try {
|
||||
spdlog::info("[{}] Simulating voice transcription...", getTestName());
|
||||
|
||||
// 1. Simulate voice transcription
|
||||
auto transcription = std::make_unique<grove::JsonDataNode>("transcription");
|
||||
transcription->setString("text", "Bonjour AISSIA, test d'intégration voice to AI");
|
||||
transcription->setDouble("confidence", 0.95);
|
||||
m_io->publish("voice:transcription", std::move(transcription));
|
||||
|
||||
// 2. Wait for AI to process and forward to LLM
|
||||
auto llmRequest = waitForMessage("llm:request", m_timeout);
|
||||
|
||||
if (!llmRequest) {
|
||||
result.passed = false;
|
||||
result.message = "Timeout waiting for llm:request (AI didn't process voice)";
|
||||
return result;
|
||||
}
|
||||
|
||||
// 3. Verify the query contains our transcription
|
||||
std::string query = llmRequest->getString("query", "");
|
||||
bool containsText = query.find("Bonjour AISSIA") != std::string::npos ||
|
||||
query.find("test") != std::string::npos;
|
||||
|
||||
result.passed = containsText;
|
||||
result.message = containsText ?
|
||||
"AI received and processed voice transcription" :
|
||||
"AI processed but query doesn't match transcription";
|
||||
result.details["query"] = query;
|
||||
|
||||
} catch (const std::exception& e) {
|
||||
result.passed = false;
|
||||
result.message = std::string("Exception: ") + e.what();
|
||||
spdlog::error("[{}] {}", getTestName(), result.message);
|
||||
}
|
||||
|
||||
auto end = std::chrono::steady_clock::now();
|
||||
result.durationMs = std::chrono::duration_cast<std::chrono::milliseconds>(
|
||||
end - start).count();
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
private:
|
||||
std::unique_ptr<grove::IDataNode> waitForMessage(
|
||||
const std::string& topic, int timeoutMs) {
|
||||
|
||||
auto start = std::chrono::steady_clock::now();
|
||||
|
||||
while (true) {
|
||||
if (m_io->hasMessages() > 0) {
|
||||
auto msg = m_io->pullMessage();
|
||||
if (msg.topic == topic && msg.data) {
|
||||
return std::move(msg.data);
|
||||
}
|
||||
}
|
||||
|
||||
auto elapsed = std::chrono::duration_cast<std::chrono::milliseconds>(
|
||||
std::chrono::steady_clock::now() - start).count();
|
||||
|
||||
if (elapsed > timeoutMs) {
|
||||
return nullptr;
|
||||
}
|
||||
|
||||
std::this_thread::sleep_for(std::chrono::milliseconds(100));
|
||||
}
|
||||
}
|
||||
|
||||
grove::IIO* m_io = nullptr;
|
||||
grove::ITaskScheduler* m_scheduler = nullptr;
|
||||
int m_timeout = 10000;
|
||||
};
|
||||
|
||||
} // namespace aissia::testing
|
||||
|
||||
extern "C" {
|
||||
grove::IModule* createModule() {
|
||||
return new aissia::testing::IT_005_VoiceToAI();
|
||||
}
|
||||
|
||||
void destroyModule(grove::IModule* module) {
|
||||
delete module;
|
||||
}
|
||||
}
|
||||
183
tests/integration/IT_006_AIToLLM.cpp
Normal file
183
tests/integration/IT_006_AIToLLM.cpp
Normal file
@ -0,0 +1,183 @@
|
||||
#include <shared/testing/ITestModule.h>
|
||||
#include <grove/JsonDataNode.h>
|
||||
#include <grove/IIO.h>
|
||||
#include <spdlog/spdlog.h>
|
||||
#include <chrono>
|
||||
#include <thread>
|
||||
|
||||
namespace aissia::testing {
|
||||
|
||||
/**
|
||||
* @brief Test AI → LLM flow with real Claude API
|
||||
*
|
||||
* Workflow:
|
||||
* 1. Publish ai:query with a simple question
|
||||
* 2. Wait for llm:response from Claude API
|
||||
* 3. Verify response is coherent and not empty
|
||||
*/
|
||||
class IT_006_AIToLLM : public ITestModule {
|
||||
public:
|
||||
std::string getTestName() const override {
|
||||
return "IT_006_AIToLLM";
|
||||
}
|
||||
|
||||
std::string getDescription() const override {
|
||||
return "Test AI → LLM with real Claude API";
|
||||
}
|
||||
|
||||
void setConfiguration(const grove::IDataNode& config,
|
||||
grove::IIO* io,
|
||||
grove::ITaskScheduler* scheduler) override {
|
||||
m_io = io;
|
||||
m_scheduler = scheduler;
|
||||
m_timeout = config.getInt("timeoutMs", 30000); // 30s for API
|
||||
|
||||
grove::SubscriptionConfig subConfig;
|
||||
m_io->subscribe("llm:response", subConfig);
|
||||
m_io->subscribe("llm:error", subConfig);
|
||||
|
||||
spdlog::info("[{}] Configured", getTestName());
|
||||
}
|
||||
|
||||
void process(const grove::IDataNode& input) override {}
|
||||
void shutdown() override {}
|
||||
|
||||
const grove::IDataNode& getConfiguration() override {
|
||||
static grove::JsonDataNode config("config");
|
||||
return config;
|
||||
}
|
||||
|
||||
std::unique_ptr<grove::IDataNode> getHealthStatus() override {
|
||||
auto status = std::make_unique<grove::JsonDataNode>("health");
|
||||
status->setString("status", "healthy");
|
||||
return status;
|
||||
}
|
||||
|
||||
std::unique_ptr<grove::IDataNode> getState() override {
|
||||
return std::make_unique<grove::JsonDataNode>("state");
|
||||
}
|
||||
|
||||
void setState(const grove::IDataNode& state) override {}
|
||||
|
||||
std::string getType() const override { return "IT_006_AIToLLM"; }
|
||||
int getVersion() const override { return 1; }
|
||||
bool isIdle() const override { return true; }
|
||||
|
||||
TestResult execute() override {
|
||||
auto start = std::chrono::steady_clock::now();
|
||||
TestResult result;
|
||||
result.testName = getTestName();
|
||||
|
||||
try {
|
||||
spdlog::info("[{}] Sending query to Claude API...", getTestName());
|
||||
|
||||
// 1. Send simple question to AI
|
||||
auto request = std::make_unique<grove::JsonDataNode>("request");
|
||||
request->setString("query", "Réponds simplement 'OK' si tu me reçois bien.");
|
||||
request->setString("conversationId", "it006");
|
||||
m_io->publish("ai:query", std::move(request));
|
||||
|
||||
// 2. Wait for LLM response
|
||||
auto response = waitForMessage("llm:response", m_timeout);
|
||||
|
||||
if (!response) {
|
||||
// Check for error
|
||||
auto error = waitForMessage("llm:error", 1000);
|
||||
if (error) {
|
||||
result.passed = false;
|
||||
result.message = "LLM error: " + error->getString("message", "Unknown");
|
||||
result.details["error"] = error->getString("message", "");
|
||||
} else {
|
||||
result.passed = false;
|
||||
result.message = "Timeout waiting for llm:response (30s)";
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
// 3. Validate response
|
||||
std::string text = response->getString("text", "");
|
||||
std::string conversationId = response->getString("conversationId", "");
|
||||
|
||||
if (text.empty()) {
|
||||
result.passed = false;
|
||||
result.message = "Empty response from LLM";
|
||||
return result;
|
||||
}
|
||||
|
||||
if (conversationId != "it006") {
|
||||
result.passed = false;
|
||||
result.message = "ConversationId mismatch";
|
||||
result.details["expected"] = "it006";
|
||||
result.details["actual"] = conversationId;
|
||||
return result;
|
||||
}
|
||||
|
||||
// Check response makes sense (contains "OK" or similar positive response)
|
||||
bool coherent = text.find("OK") != std::string::npos ||
|
||||
text.find("oui") != std::string::npos ||
|
||||
text.find("Oui") != std::string::npos ||
|
||||
text.find("reçois") != std::string::npos;
|
||||
|
||||
result.passed = coherent;
|
||||
result.message = coherent ?
|
||||
"LLM response received and coherent" :
|
||||
"LLM responded but answer unexpected";
|
||||
result.details["response"] = text;
|
||||
result.details["responseLength"] = static_cast<int>(text.length());
|
||||
|
||||
spdlog::info("[{}] Claude response: {}", getTestName(), text.substr(0, 100));
|
||||
|
||||
} catch (const std::exception& e) {
|
||||
result.passed = false;
|
||||
result.message = std::string("Exception: ") + e.what();
|
||||
spdlog::error("[{}] {}", getTestName(), result.message);
|
||||
}
|
||||
|
||||
auto end = std::chrono::steady_clock::now();
|
||||
result.durationMs = std::chrono::duration_cast<std::chrono::milliseconds>(
|
||||
end - start).count();
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
private:
|
||||
std::unique_ptr<grove::IDataNode> waitForMessage(
|
||||
const std::string& topic, int timeoutMs) {
|
||||
|
||||
auto start = std::chrono::steady_clock::now();
|
||||
|
||||
while (true) {
|
||||
if (m_io->hasMessages() > 0) {
|
||||
auto msg = m_io->pullMessage();
|
||||
if (msg.topic == topic && msg.data) {
|
||||
return std::move(msg.data);
|
||||
}
|
||||
}
|
||||
|
||||
auto elapsed = std::chrono::duration_cast<std::chrono::milliseconds>(
|
||||
std::chrono::steady_clock::now() - start).count();
|
||||
|
||||
if (elapsed > timeoutMs) {
|
||||
return nullptr;
|
||||
}
|
||||
|
||||
std::this_thread::sleep_for(std::chrono::milliseconds(100));
|
||||
}
|
||||
}
|
||||
|
||||
grove::IIO* m_io = nullptr;
|
||||
grove::ITaskScheduler* m_scheduler = nullptr;
|
||||
int m_timeout = 30000;
|
||||
};
|
||||
|
||||
} // namespace aissia::testing
|
||||
|
||||
extern "C" {
|
||||
grove::IModule* createModule() {
|
||||
return new aissia::testing::IT_006_AIToLLM();
|
||||
}
|
||||
|
||||
void destroyModule(grove::IModule* module) {
|
||||
delete module;
|
||||
}
|
||||
}
|
||||
196
tests/integration/IT_007_StorageWrite.cpp
Normal file
196
tests/integration/IT_007_StorageWrite.cpp
Normal file
@ -0,0 +1,196 @@
|
||||
#include <shared/testing/ITestModule.h>
|
||||
#include <grove/JsonDataNode.h>
|
||||
#include <grove/IIO.h>
|
||||
#include <spdlog/spdlog.h>
|
||||
#include <chrono>
|
||||
#include <thread>
|
||||
#include <filesystem>
|
||||
#include <fstream>
|
||||
|
||||
namespace fs = std::filesystem;
|
||||
|
||||
namespace aissia::testing {
|
||||
|
||||
/**
|
||||
* @brief Test Storage write via AI tool
|
||||
*
|
||||
* Workflow:
|
||||
* 1. Ask AI to save a note using storage_save_note tool
|
||||
* 2. Wait for LLM response confirming save
|
||||
* 3. Verify note file was created in data/notes/
|
||||
*/
|
||||
class IT_007_StorageWrite : public ITestModule {
|
||||
public:
|
||||
std::string getTestName() const override {
|
||||
return "IT_007_StorageWrite";
|
||||
}
|
||||
|
||||
std::string getDescription() const override {
|
||||
return "Test Storage write via AI tool";
|
||||
}
|
||||
|
||||
void setConfiguration(const grove::IDataNode& config,
|
||||
grove::IIO* io,
|
||||
grove::ITaskScheduler* scheduler) override {
|
||||
m_io = io;
|
||||
m_scheduler = scheduler;
|
||||
m_timeout = config.getInt("timeoutMs", 30000);
|
||||
|
||||
grove::SubscriptionConfig subConfig;
|
||||
m_io->subscribe("llm:response", subConfig);
|
||||
m_io->subscribe("llm:error", subConfig);
|
||||
|
||||
spdlog::info("[{}] Configured", getTestName());
|
||||
}
|
||||
|
||||
void process(const grove::IDataNode& input) override {}
|
||||
void shutdown() override {}
|
||||
|
||||
const grove::IDataNode& getConfiguration() override {
|
||||
static grove::JsonDataNode config("config");
|
||||
return config;
|
||||
}
|
||||
|
||||
std::unique_ptr<grove::IDataNode> getHealthStatus() override {
|
||||
auto status = std::make_unique<grove::JsonDataNode>("health");
|
||||
status->setString("status", "healthy");
|
||||
return status;
|
||||
}
|
||||
|
||||
std::unique_ptr<grove::IDataNode> getState() override {
|
||||
return std::make_unique<grove::JsonDataNode>("state");
|
||||
}
|
||||
|
||||
void setState(const grove::IDataNode& state) override {}
|
||||
|
||||
std::string getType() const override { return "IT_007_StorageWrite"; }
|
||||
int getVersion() const override { return 1; }
|
||||
bool isIdle() const override { return true; }
|
||||
|
||||
TestResult execute() override {
|
||||
auto start = std::chrono::steady_clock::now();
|
||||
TestResult result;
|
||||
result.testName = getTestName();
|
||||
|
||||
const std::string noteContent = "Test IT_007: Integration test storage write";
|
||||
|
||||
try {
|
||||
spdlog::info("[{}] Asking AI to save note...", getTestName());
|
||||
|
||||
// 1. Ask AI to save a note
|
||||
auto request = std::make_unique<grove::JsonDataNode>("request");
|
||||
request->setString("query",
|
||||
"Utilise le tool storage_save_note pour sauvegarder cette note: '" +
|
||||
noteContent + "' avec le titre 'Integration Test IT007'");
|
||||
request->setString("conversationId", "it007");
|
||||
m_io->publish("ai:query", std::move(request));
|
||||
|
||||
// 2. Wait for response
|
||||
auto response = waitForMessage("llm:response", m_timeout);
|
||||
|
||||
if (!response) {
|
||||
auto error = waitForMessage("llm:error", 1000);
|
||||
if (error) {
|
||||
result.passed = false;
|
||||
result.message = "LLM error: " + error->getString("message", "Unknown");
|
||||
} else {
|
||||
result.passed = false;
|
||||
result.message = "Timeout waiting for llm:response";
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
std::string text = response->getString("text", "");
|
||||
spdlog::info("[{}] LLM response: {}", getTestName(), text);
|
||||
|
||||
// 3. Give time for async storage operation
|
||||
std::this_thread::sleep_for(std::chrono::milliseconds(2000));
|
||||
|
||||
// 4. Check if note was saved (look for any .md file with our content)
|
||||
bool noteFound = false;
|
||||
std::string foundPath;
|
||||
|
||||
if (fs::exists("data/notes")) {
|
||||
for (const auto& entry : fs::recursive_directory_iterator("data/notes")) {
|
||||
if (entry.is_regular_file() && entry.path().extension() == ".md") {
|
||||
std::ifstream file(entry.path());
|
||||
std::string content((std::istreambuf_iterator<char>(file)),
|
||||
std::istreambuf_iterator<char>());
|
||||
|
||||
if (content.find("IT_007") != std::string::npos) {
|
||||
noteFound = true;
|
||||
foundPath = entry.path().string();
|
||||
|
||||
// Cleanup
|
||||
file.close();
|
||||
fs::remove(entry.path());
|
||||
break;
|
||||
}
|
||||
file.close();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
result.passed = noteFound;
|
||||
result.message = noteFound ?
|
||||
"Note saved successfully" :
|
||||
"Note not found in data/notes/";
|
||||
result.details["response"] = text;
|
||||
if (noteFound) {
|
||||
result.details["notePath"] = foundPath;
|
||||
}
|
||||
|
||||
} catch (const std::exception& e) {
|
||||
result.passed = false;
|
||||
result.message = std::string("Exception: ") + e.what();
|
||||
spdlog::error("[{}] {}", getTestName(), result.message);
|
||||
}
|
||||
|
||||
auto end = std::chrono::steady_clock::now();
|
||||
result.durationMs = std::chrono::duration_cast<std::chrono::milliseconds>(
|
||||
end - start).count();
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
private:
|
||||
std::unique_ptr<grove::IDataNode> waitForMessage(
|
||||
const std::string& topic, int timeoutMs) {
|
||||
|
||||
auto start = std::chrono::steady_clock::now();
|
||||
|
||||
while (true) {
|
||||
if (m_io->hasMessages() > 0) {
|
||||
auto msg = m_io->pullMessage();
|
||||
if (msg.topic == topic && msg.data) {
|
||||
return std::move(msg.data);
|
||||
}
|
||||
}
|
||||
|
||||
auto elapsed = std::chrono::duration_cast<std::chrono::milliseconds>(
|
||||
std::chrono::steady_clock::now() - start).count();
|
||||
|
||||
if (elapsed > timeoutMs) {
|
||||
return nullptr;
|
||||
}
|
||||
|
||||
std::this_thread::sleep_for(std::chrono::milliseconds(100));
|
||||
}
|
||||
}
|
||||
|
||||
grove::IIO* m_io = nullptr;
|
||||
grove::ITaskScheduler* m_scheduler = nullptr;
|
||||
int m_timeout = 30000;
|
||||
};
|
||||
|
||||
} // namespace aissia::testing
|
||||
|
||||
extern "C" {
|
||||
grove::IModule* createModule() {
|
||||
return new aissia::testing::IT_007_StorageWrite();
|
||||
}
|
||||
|
||||
void destroyModule(grove::IModule* module) {
|
||||
delete module;
|
||||
}
|
||||
}
|
||||
186
tests/integration/IT_008_StorageRead.cpp
Normal file
186
tests/integration/IT_008_StorageRead.cpp
Normal file
@ -0,0 +1,186 @@
|
||||
#include <shared/testing/ITestModule.h>
|
||||
#include <grove/JsonDataNode.h>
|
||||
#include <grove/IIO.h>
|
||||
#include <spdlog/spdlog.h>
|
||||
#include <chrono>
|
||||
#include <thread>
|
||||
#include <filesystem>
|
||||
#include <fstream>
|
||||
|
||||
namespace fs = std::filesystem;
|
||||
|
||||
namespace aissia::testing {
|
||||
|
||||
/**
|
||||
* @brief Test Storage read via AI tool
|
||||
*
|
||||
* Workflow:
|
||||
* 1. Create a test note file
|
||||
* 2. Ask AI to query notes using storage_query_notes
|
||||
* 3. Verify AI returns the note content
|
||||
*/
|
||||
class IT_008_StorageRead : public ITestModule {
|
||||
public:
|
||||
std::string getTestName() const override {
|
||||
return "IT_008_StorageRead";
|
||||
}
|
||||
|
||||
std::string getDescription() const override {
|
||||
return "Test Storage read via AI tool";
|
||||
}
|
||||
|
||||
void setConfiguration(const grove::IDataNode& config,
|
||||
grove::IIO* io,
|
||||
grove::ITaskScheduler* scheduler) override {
|
||||
m_io = io;
|
||||
m_scheduler = scheduler;
|
||||
m_timeout = config.getInt("timeoutMs", 30000);
|
||||
|
||||
grove::SubscriptionConfig subConfig;
|
||||
m_io->subscribe("llm:response", subConfig);
|
||||
m_io->subscribe("llm:error", subConfig);
|
||||
|
||||
spdlog::info("[{}] Configured", getTestName());
|
||||
}
|
||||
|
||||
void process(const grove::IDataNode& input) override {}
|
||||
void shutdown() override {}
|
||||
|
||||
const grove::IDataNode& getConfiguration() override {
|
||||
static grove::JsonDataNode config("config");
|
||||
return config;
|
||||
}
|
||||
|
||||
std::unique_ptr<grove::IDataNode> getHealthStatus() override {
|
||||
auto status = std::make_unique<grove::JsonDataNode>("health");
|
||||
status->setString("status", "healthy");
|
||||
return status;
|
||||
}
|
||||
|
||||
std::unique_ptr<grove::IDataNode> getState() override {
|
||||
return std::make_unique<grove::JsonDataNode>("state");
|
||||
}
|
||||
|
||||
void setState(const grove::IDataNode& state) override {}
|
||||
|
||||
std::string getType() const override { return "IT_008_StorageRead"; }
|
||||
int getVersion() const override { return 1; }
|
||||
bool isIdle() const override { return true; }
|
||||
|
||||
TestResult execute() override {
|
||||
auto start = std::chrono::steady_clock::now();
|
||||
TestResult result;
|
||||
result.testName = getTestName();
|
||||
|
||||
const std::string testContent = "IT_008 test data for storage read";
|
||||
const std::string testFile = "data/notes/test_it008.md";
|
||||
|
||||
try {
|
||||
// 1. Create test note
|
||||
fs::create_directories("data/notes");
|
||||
std::ofstream file(testFile);
|
||||
file << "# Test Note IT008\n\n" << testContent;
|
||||
file.close();
|
||||
spdlog::info("[{}] Created test note: {}", getTestName(), testFile);
|
||||
|
||||
// 2. Ask AI to query notes
|
||||
spdlog::info("[{}] Asking AI to query notes...", getTestName());
|
||||
auto request = std::make_unique<grove::JsonDataNode>("request");
|
||||
request->setString("query",
|
||||
"Utilise le tool storage_query_notes pour chercher les notes contenant 'IT_008'");
|
||||
request->setString("conversationId", "it008");
|
||||
m_io->publish("ai:query", std::move(request));
|
||||
|
||||
// 3. Wait for response
|
||||
auto response = waitForMessage("llm:response", m_timeout);
|
||||
|
||||
if (!response) {
|
||||
auto error = waitForMessage("llm:error", 1000);
|
||||
if (error) {
|
||||
result.passed = false;
|
||||
result.message = "LLM error: " + error->getString("message", "Unknown");
|
||||
} else {
|
||||
result.passed = false;
|
||||
result.message = "Timeout waiting for llm:response";
|
||||
}
|
||||
// Cleanup
|
||||
if (fs::exists(testFile)) fs::remove(testFile);
|
||||
return result;
|
||||
}
|
||||
|
||||
std::string text = response->getString("text", "");
|
||||
spdlog::info("[{}] LLM response: {}", getTestName(), text);
|
||||
|
||||
// 4. Verify response contains our test data
|
||||
bool contentFound = text.find("IT_008") != std::string::npos ||
|
||||
text.find("test data") != std::string::npos;
|
||||
|
||||
result.passed = contentFound;
|
||||
result.message = contentFound ?
|
||||
"Note retrieved successfully" :
|
||||
"Note content not found in response";
|
||||
result.details["response"] = text;
|
||||
result.details["expectedContent"] = testContent;
|
||||
|
||||
// Cleanup
|
||||
if (fs::exists(testFile)) {
|
||||
fs::remove(testFile);
|
||||
}
|
||||
|
||||
} catch (const std::exception& e) {
|
||||
result.passed = false;
|
||||
result.message = std::string("Exception: ") + e.what();
|
||||
spdlog::error("[{}] {}", getTestName(), result.message);
|
||||
|
||||
// Cleanup on error
|
||||
if (fs::exists(testFile)) fs::remove(testFile);
|
||||
}
|
||||
|
||||
auto end = std::chrono::steady_clock::now();
|
||||
result.durationMs = std::chrono::duration_cast<std::chrono::milliseconds>(
|
||||
end - start).count();
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
private:
|
||||
std::unique_ptr<grove::IDataNode> waitForMessage(
|
||||
const std::string& topic, int timeoutMs) {
|
||||
|
||||
auto start = std::chrono::steady_clock::now();
|
||||
|
||||
while (true) {
|
||||
if (m_io->hasMessages() > 0) {
|
||||
auto msg = m_io->pullMessage();
|
||||
if (msg.topic == topic && msg.data) {
|
||||
return std::move(msg.data);
|
||||
}
|
||||
}
|
||||
|
||||
auto elapsed = std::chrono::duration_cast<std::chrono::milliseconds>(
|
||||
std::chrono::steady_clock::now() - start).count();
|
||||
|
||||
if (elapsed > timeoutMs) {
|
||||
return nullptr;
|
||||
}
|
||||
|
||||
std::this_thread::sleep_for(std::chrono::milliseconds(100));
|
||||
}
|
||||
}
|
||||
|
||||
grove::IIO* m_io = nullptr;
|
||||
grove::ITaskScheduler* m_scheduler = nullptr;
|
||||
int m_timeout = 30000;
|
||||
};
|
||||
|
||||
} // namespace aissia::testing
|
||||
|
||||
extern "C" {
|
||||
grove::IModule* createModule() {
|
||||
return new aissia::testing::IT_008_StorageRead();
|
||||
}
|
||||
|
||||
void destroyModule(grove::IModule* module) {
|
||||
delete module;
|
||||
}
|
||||
}
|
||||
349
tests/integration/README.md
Normal file
349
tests/integration/README.md
Normal file
@ -0,0 +1,349 @@
|
||||
# AISSIA Integration Tests
|
||||
|
||||
Système de tests d'intégration modulaires pour valider AISSIA en conditions réelles.
|
||||
|
||||
## Architecture
|
||||
|
||||
**Philosophie : "Un module = Un test"**
|
||||
|
||||
Chaque test d'intégration est un module GroveEngine indépendant (`.so`) :
|
||||
- Chargé dynamiquement par le TestRunnerModule
|
||||
- Exécute un scénario spécifique
|
||||
- Retourne un résultat (pass/fail + détails)
|
||||
- Peut être modifié et hot-reload sans tout recompiler
|
||||
|
||||
```
|
||||
AISSIA --run-tests
|
||||
├─ TestRunnerModule (orchestrateur)
|
||||
│ ├─ Découvre tests/integration/IT_*.so
|
||||
│ ├─ Pour chaque test:
|
||||
│ │ ├─ Charge dynamiquement
|
||||
│ │ ├─ Execute via execute()
|
||||
│ │ ├─ Collecte résultat
|
||||
│ │ └─ Décharge
|
||||
│ ├─ Génère rapport console
|
||||
│ ├─ Génère rapport JSON
|
||||
│ └─ Exit(0|1)
|
||||
│
|
||||
└─ Tests d'intégration (.so)
|
||||
├─ IT_001_GetCurrentTime
|
||||
├─ IT_002_FileSystemWrite
|
||||
├─ IT_003_FileSystemRead
|
||||
├─ IT_004_MCPToolsList
|
||||
├─ IT_005_VoiceToAI
|
||||
├─ IT_006_AIToLLM
|
||||
├─ IT_007_StorageWrite
|
||||
└─ IT_008_StorageRead
|
||||
```
|
||||
|
||||
## Tests Disponibles
|
||||
|
||||
### Phase 1: Tests MCP (Tools)
|
||||
|
||||
| Test | Description | Durée |
|
||||
|------|-------------|-------|
|
||||
| **IT_001_GetCurrentTime** | Test tool `get_current_time` via AI | ~2s |
|
||||
| **IT_002_FileSystemWrite** | Test tool `filesystem_write` | ~3s |
|
||||
| **IT_003_FileSystemRead** | Test tool `filesystem_read` | ~3s |
|
||||
| **IT_004_MCPToolsList** | Vérification inventaire tools (≥5) | ~3s |
|
||||
|
||||
### Phase 2: Tests Flux
|
||||
|
||||
| Test | Description | Durée |
|
||||
|------|-------------|-------|
|
||||
| **IT_005_VoiceToAI** | Communication Voice → AI | ~2s |
|
||||
| **IT_006_AIToLLM** | Requête AI → Claude API (réelle) | ~5s |
|
||||
| **IT_007_StorageWrite** | AI → Storage (sauvegarde note) | ~4s |
|
||||
| **IT_008_StorageRead** | AI → Storage (lecture note) | ~4s |
|
||||
|
||||
## Utilisation
|
||||
|
||||
### Build
|
||||
|
||||
```bash
|
||||
# Build complet
|
||||
cmake -B build -DBUILD_TESTING=ON
|
||||
cmake --build build -j4
|
||||
|
||||
# Build seulement tests d'intégration
|
||||
cmake --build build --target integration_tests -j4
|
||||
```
|
||||
|
||||
### Exécution
|
||||
|
||||
```bash
|
||||
# Lancer tous les tests
|
||||
cd build && ./aissia --run-tests
|
||||
|
||||
# Résultats
|
||||
# - Console: rapport détaillé avec ✅/❌
|
||||
# - JSON: test-results.json avec détails complets
|
||||
# - Exit code: 0 = success, 1 = au moins un échec
|
||||
```
|
||||
|
||||
### Exemple de sortie
|
||||
|
||||
```
|
||||
========================================
|
||||
AISSIA Integration Tests
|
||||
Running 8 test(s)...
|
||||
========================================
|
||||
|
||||
[1/8] IT_001_GetCurrentTime............ ✅ PASS (1.8s)
|
||||
Tool returned valid time
|
||||
|
||||
[2/8] IT_002_FileSystemWrite........... ✅ PASS (2.3s)
|
||||
File created with correct content
|
||||
|
||||
[3/8] IT_003_FileSystemRead............ ✅ PASS (1.9s)
|
||||
Content read correctly
|
||||
|
||||
[4/8] IT_004_MCPToolsList.............. ✅ PASS (3.1s)
|
||||
Found 9 tools in response
|
||||
|
||||
[5/8] IT_005_VoiceToAI................. ✅ PASS (1.5s)
|
||||
AI received and processed voice transcription
|
||||
|
||||
[6/8] IT_006_AIToLLM................... ✅ PASS (4.7s)
|
||||
LLM response received and coherent
|
||||
|
||||
[7/8] IT_007_StorageWrite.............. ✅ PASS (3.8s)
|
||||
Note saved successfully
|
||||
|
||||
[8/8] IT_008_StorageRead............... ✅ PASS (3.2s)
|
||||
Note retrieved successfully
|
||||
|
||||
========================================
|
||||
Results: 8/8 passed (100%)
|
||||
Total time: 22.3s
|
||||
========================================
|
||||
|
||||
Exit code: 0
|
||||
```
|
||||
|
||||
## Configuration
|
||||
|
||||
### Global : `config/test_runner.json`
|
||||
|
||||
```json
|
||||
{
|
||||
"enabled": true,
|
||||
"testDirectory": "tests/integration",
|
||||
"globalTimeoutMs": 300000,
|
||||
"stopOnFirstFailure": false,
|
||||
"verboseOutput": true,
|
||||
"jsonOutputPath": "test-results.json"
|
||||
}
|
||||
```
|
||||
|
||||
### Par test : `config/integration/IT_XXX.json`
|
||||
|
||||
```json
|
||||
{
|
||||
"enabled": true,
|
||||
"timeoutMs": 10000,
|
||||
"retryCount": 0,
|
||||
"description": "Test description",
|
||||
"tags": ["mcp", "tools", "quick"]
|
||||
}
|
||||
```
|
||||
|
||||
## Ajouter un nouveau test
|
||||
|
||||
1. **Créer le module** : `tests/integration/IT_XXX_TestName.cpp`
|
||||
|
||||
```cpp
|
||||
#include <shared/testing/ITestModule.h>
|
||||
#include <grove/JsonDataNode.h>
|
||||
#include <grove/IIO.h>
|
||||
|
||||
namespace aissia::testing {
|
||||
|
||||
class IT_XXX_TestName : public ITestModule {
|
||||
public:
|
||||
std::string getTestName() const override {
|
||||
return "IT_XXX_TestName";
|
||||
}
|
||||
|
||||
std::string getDescription() const override {
|
||||
return "Description du test";
|
||||
}
|
||||
|
||||
// Implémenter les méthodes virtuelles de IModule
|
||||
void setConfiguration(...) override { ... }
|
||||
void process(...) override {}
|
||||
void shutdown() override {}
|
||||
const IDataNode& getConfiguration() override { ... }
|
||||
std::unique_ptr<IDataNode> getHealthStatus() override { ... }
|
||||
std::unique_ptr<IDataNode> getState() override { ... }
|
||||
void setState(...) override {}
|
||||
std::string getType() const override { return "IT_XXX_TestName"; }
|
||||
int getVersion() const override { return 1; }
|
||||
bool isIdle() const override { return true; }
|
||||
|
||||
// Logique du test
|
||||
TestResult execute() override {
|
||||
TestResult result;
|
||||
result.testName = getTestName();
|
||||
|
||||
try {
|
||||
// 1. Envoyer requête IIO
|
||||
auto request = std::make_unique<JsonDataNode>("request");
|
||||
request->setString("query", "...");
|
||||
m_io->publish("topic:name", std::move(request));
|
||||
|
||||
// 2. Attendre réponse
|
||||
auto response = waitForMessage("topic:response", timeout);
|
||||
|
||||
// 3. Valider
|
||||
result.passed = /* validation */;
|
||||
result.message = "...";
|
||||
|
||||
} catch (const std::exception& e) {
|
||||
result.passed = false;
|
||||
result.message = e.what();
|
||||
}
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
private:
|
||||
std::unique_ptr<IDataNode> waitForMessage(
|
||||
const std::string& topic, int timeoutMs) {
|
||||
// Polling avec timeout
|
||||
}
|
||||
|
||||
IIO* m_io = nullptr;
|
||||
};
|
||||
|
||||
} // namespace aissia::testing
|
||||
|
||||
extern "C" {
|
||||
grove::IModule* createModule() {
|
||||
return new aissia::testing::IT_XXX_TestName();
|
||||
}
|
||||
|
||||
void destroyModule(grove::IModule* module) {
|
||||
delete module;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
2. **Ajouter au CMakeLists.txt** : `tests/CMakeLists.txt`
|
||||
|
||||
```cmake
|
||||
add_integration_test(IT_XXX_TestName)
|
||||
|
||||
add_custom_target(integration_tests
|
||||
DEPENDS
|
||||
...
|
||||
IT_XXX_TestName
|
||||
...
|
||||
)
|
||||
```
|
||||
|
||||
3. **Build et test**
|
||||
|
||||
```bash
|
||||
cmake --build build --target integration_tests
|
||||
cd build && ./aissia --run-tests
|
||||
```
|
||||
|
||||
## Notes Importantes
|
||||
|
||||
### Conditions Réelles
|
||||
|
||||
Les tests utilisent de **vraies ressources** :
|
||||
- **Claude API** : Requêtes réelles (coût tokens)
|
||||
- **Fichiers** : Écriture dans `data/` (cleanup auto)
|
||||
- **Réseau** : Requêtes HTTP réelles
|
||||
|
||||
⚠️ Les tests peuvent échouer si :
|
||||
- API Claude down ou clé invalide
|
||||
- Problèmes réseau
|
||||
- Disk plein
|
||||
|
||||
### Timeouts
|
||||
|
||||
- Tests MCP simples : 5-10s
|
||||
- Tests LLM : 30s (Claude peut être lent)
|
||||
- Test complet (IT_009) : 60s
|
||||
|
||||
### Isolation
|
||||
|
||||
Chaque test :
|
||||
- Ne pollue pas les autres
|
||||
- Cleanup automatique des fichiers temporaires
|
||||
- ConversationId unique pour éviter les collisions
|
||||
|
||||
### CI/CD
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
# ci-test.sh
|
||||
|
||||
cmake -B build -DBUILD_TESTING=ON
|
||||
cmake --build build --target integration_tests -j4
|
||||
|
||||
./build/aissia --run-tests --json-output results.json
|
||||
|
||||
# Exit code: 0 = success, 1 = failure
|
||||
exit $?
|
||||
```
|
||||
|
||||
## Développement
|
||||
|
||||
### Structure du code
|
||||
|
||||
```
|
||||
tests/
|
||||
├── integration/
|
||||
│ ├── IT_001_GetCurrentTime.cpp
|
||||
│ ├── IT_002_FileSystemWrite.cpp
|
||||
│ └── ...
|
||||
├── CMakeLists.txt
|
||||
└── README.md (ce fichier)
|
||||
|
||||
src/
|
||||
├── shared/
|
||||
│ └── testing/
|
||||
│ └── ITestModule.h # Interface de base
|
||||
└── modules/
|
||||
└── TestRunnerModule.{h,cpp} # Orchestrateur
|
||||
|
||||
config/
|
||||
├── test_runner.json
|
||||
└── integration/
|
||||
├── IT_001.json
|
||||
└── ...
|
||||
```
|
||||
|
||||
### Bonnes pratiques
|
||||
|
||||
1. **Nommage** : `IT_XXX_DescriptiveName` (XXX = numéro séquentiel)
|
||||
2. **Taille** : ~150-200 lignes par test max
|
||||
3. **Timeout** : Toujours utiliser un timeout raisonnable
|
||||
4. **Cleanup** : Supprimer les fichiers/données temporaires
|
||||
5. **Logs** : Utiliser `spdlog::info` pour debugging
|
||||
6. **Erreurs** : Toujours catcher les exceptions
|
||||
|
||||
## Roadmap
|
||||
|
||||
### Implémenté ✅
|
||||
|
||||
- [x] Infrastructure (ITestModule, TestRunnerModule)
|
||||
- [x] Tests MCP (IT_001-004)
|
||||
- [x] Tests Flux (IT_005-008)
|
||||
|
||||
### À venir 📋
|
||||
|
||||
- [ ] Test end-to-end (IT_009_FullConversationLoop)
|
||||
- [ ] Tests modules (IT_010-013: Scheduler, Notification, Monitoring, Web)
|
||||
- [ ] Tests avancés (hot-reload, charge, récupération d'erreur)
|
||||
- [ ] Dashboard web pour visualisation des résultats
|
||||
|
||||
---
|
||||
|
||||
**Auteur** : Claude Code
|
||||
**Date** : 2025-11-28
|
||||
**Version** : 1.0.0
|
||||
Loading…
Reference in New Issue
Block a user