feat(query-optimization): implement GraphQL query optimization and enhance regression suite
- **Optimization**: Implemented automatic Zabbix parameter optimization by analyzing GraphQL selection sets. - **ZabbixRequest**: Added optimizeZabbixParams with support for skippable parameters and implied field dependencies (e.g., state -> items). - **Resolvers**: Updated allHosts, allDevices, allHostGroups, and templates to pass requested fields to data sources. - **Data Sources**: Optimized ZabbixQueryHostsGenericRequest and ZabbixQueryTemplatesRequest to skip unnecessary Zabbix API calls. - **Regression Tests**: Enhanced RegressionTestExecutor with new tests for optimization (REG-OPT, REG-OPT-NEG), state retrieval (REG-STATE), dependent items (REG-DEP), and empty results (REG-EMPTY). - **Documentation**: Created query_optimization.md How-To guide and updated roadmap.md, README.md, and tests.md. - **Bug Fixes**: Fixed deviceType tag assignment during host import and corrected ZabbixCreateHostRequest to support tags.
This commit is contained in:
parent
ad104acde2
commit
97a0f70fd6
16 changed files with 835 additions and 69 deletions
|
|
@ -7,6 +7,9 @@ This directory contains detailed guides on how to use and extend the Zabbix Grap
|
|||
### 🍳 [Cookbook](./cookbook.md)
|
||||
Practical, step-by-step recipes for common tasks, designed for both humans and AI-based test generation.
|
||||
|
||||
### ⚡ [Query Optimization](./query_optimization.md)
|
||||
Learn how the API optimizes Zabbix requests by reducing output fields and skipping unnecessary parameters based on the GraphQL query.
|
||||
|
||||
### 📊 [Schema and Schema Extension](./schema.md)
|
||||
Learn about the GraphQL schema structure, how Zabbix entities map to GraphQL types, and how to use the dynamic schema extension system.
|
||||
|
||||
|
|
|
|||
51
docs/howtos/query_optimization.md
Normal file
51
docs/howtos/query_optimization.md
Normal file
|
|
@ -0,0 +1,51 @@
|
|||
# ⚡ Query Optimization
|
||||
|
||||
This document describes how the Zabbix GraphQL API optimizes queries to reduce data fetching from Zabbix, improving performance and reducing network load.
|
||||
|
||||
## 🚀 Overview
|
||||
|
||||
The optimization works by analyzing the requested GraphQL fields and only fetching the necessary data from the Zabbix API. This is achieved through:
|
||||
- **Output Reduction**: Dynamically setting the `output` parameter in Zabbix requests.
|
||||
- **Parameter Skipping**: Automatically removing optional Zabbix parameters (like `selectTags` or `selectItems`) if the corresponding GraphQL fields are not requested.
|
||||
|
||||
## 🏗️ Implementation Details
|
||||
|
||||
### 1. `ZabbixRequest` Enhancement
|
||||
The base `ZabbixRequest` class handles the core optimization logic:
|
||||
- `optimizeZabbixParams(params, output)`: This method modifies the Zabbix parameters. It filters the `output` array to match the requested fields and removes "skippable" parameters based on rules.
|
||||
- `skippableZabbixParams`: A map that defines dependencies between Zabbix parameters and GraphQL fields.
|
||||
- *Example*: `this.skippableZabbixParams.set("selectTags", "tags")` means `selectTags` will be removed if `tags` is not in the requested output.
|
||||
|
||||
### 2. Parameter Mapping
|
||||
The `GraphqlParamsToNeededZabbixOutput` class provides static methods to map GraphQL query arguments and the selection set (`GraphQLResolveInfo`) to a list of needed Zabbix fields.
|
||||
|
||||
### 3. Resolver Integration
|
||||
Resolvers use the mapper to determine the required output and pass it to the datasource:
|
||||
```typescript
|
||||
const output = GraphqlParamsToNeededZabbixOutput.mapAllHosts(args, info);
|
||||
return await new ZabbixQueryHostsRequestWithItemsAndInventory(...)
|
||||
.executeRequestThrowError(dataSources.zabbixAPI, new ParsedArgs(args), output);
|
||||
```
|
||||
|
||||
### 4. Indirect Dependencies
|
||||
Some GraphQL fields are not directly returned by Zabbix but are computed from other data. The optimization logic ensures these dependencies are handled:
|
||||
- **`state`**: Requesting the `state` field on a `Device` requires Zabbix `items`. The mapper automatically adds `items` to the requested output if `state` is present.
|
||||
|
||||
## 🛠️ Configuration
|
||||
Optimization rules are defined in the constructor of specialized `ZabbixRequest` classes.
|
||||
|
||||
### 📋 Supported Optimizations
|
||||
- **Hosts & Devices**:
|
||||
- `selectParentTemplates` skipped if `parentTemplates` not requested.
|
||||
- `selectTags` and `selectInheritedTags` skipped if `tags` not requested.
|
||||
- `selectHostGroups` skipped if `hostgroups` not requested.
|
||||
- `selectItems` skipped if `items` (or `state`) not requested.
|
||||
- `selectInventory` skipped if `inventory` not requested.
|
||||
- **Templates**:
|
||||
- `selectItems` skipped if `items` not requested.
|
||||
|
||||
## ✅ Verification
|
||||
You can verify that optimization is working by:
|
||||
1. Enabling `debug` log level (`LOG_LEVEL=debug`).
|
||||
2. Observing the Zabbix request bodies in the logs.
|
||||
3. Checking that the `output` array is minimized and `select*` parameters are omitted when not needed.
|
||||
|
|
@ -53,6 +53,11 @@ This document outlines the test cases and coverage for the Zabbix GraphQL API.
|
|||
- **TC-AUTH-04**: Import user rights.
|
||||
- **TC-AUTH-05**: Import user rights using sample mutation.
|
||||
|
||||
### Query Optimization
|
||||
- **TC-OPT-01**: Verify that GraphQL queries only fetch requested fields from Zabbix (reduced output).
|
||||
- **TC-OPT-02**: Verify that skippable Zabbix parameters (like selectItems) are omitted if not requested in GraphQL.
|
||||
- **TC-OPT-03**: Verify that indirect dependencies (e.g., `state` requiring `items`) are correctly handled by the optimization logic.
|
||||
|
||||
### System and Configuration
|
||||
- **TC-CONF-01**: Schema loader uses Config variables.
|
||||
- **TC-CONF-02**: Zabbix API constants derived from Config.
|
||||
|
|
@ -77,6 +82,11 @@ The `runAllRegressionTests` mutation (TC-E2E-02) executes the following checks:
|
|||
- **Template technical name lookup**: Verifies that templates can be correctly identified by their technical name (`host` field) when linking them to hosts during import.
|
||||
- **HTTP Agent URL support**: Verifies that templates containing HTTP Agent items with a configured URL can be imported successfully (verifying the addition of the `url` field to `CreateTemplateItem`).
|
||||
- **Host retrieval and visibility**: Verifies that newly imported hosts are immediately visible and retrievable via the `allHosts` query, including correctly delivered assigned templates and assigned host groups (verifying the fix for `output` fields in the host query data source).
|
||||
- **Query Optimization**: Verifies that GraphQL requests correctly translate into optimized Zabbix parameters, reducing the amount of data fetched (verifying the query optimization feature).
|
||||
- **Empty result handling**: Verifies that queries return an empty array instead of an error when no entities match the provided filters.
|
||||
- **Dependent Items**: Verifies that templates with master and dependent items can be imported successfully, correctly resolving the dependency within the same import operation.
|
||||
- **State sub-properties**: Verifies that requesting device state sub-properties correctly triggers the retrieval of required Zabbix items, even if `items` is not explicitly requested (verifying the indirect dependency logic).
|
||||
- **Negative Optimization (allDevices)**: Verifies that items are NOT requested from Zabbix if neither `items` nor `state` (or state sub-properties) are requested within the `allDevices` query.
|
||||
|
||||
## ✅ Test Coverage Checklist
|
||||
|
||||
|
|
@ -116,6 +126,9 @@ The `runAllRegressionTests` mutation (TC-E2E-02) executes the following checks:
|
|||
| TC-AUTH-03 | hasPermissions query | Unit | Jest | [src/test/user_rights.test.ts](../src/test/user_rights.test.ts) |
|
||||
| TC-AUTH-04 | importUserRights mutation | Unit | Jest | [src/test/user_rights.test.ts](../src/test/user_rights.test.ts) |
|
||||
| TC-AUTH-05 | Import user rights using sample | Integration | Jest | [src/test/user_rights_integration.test.ts](../src/test/user_rights_integration.test.ts) |
|
||||
| TC-OPT-01 | Verify Query Optimization (reduced output) | Unit/E2E | Jest/Regression | [src/test/query_optimization.test.ts](../src/test/query_optimization.test.ts) |
|
||||
| TC-OPT-02 | Verify skippable parameters | Unit/E2E | Jest/Regression | [src/test/query_optimization.test.ts](../src/test/query_optimization.test.ts) |
|
||||
| TC-OPT-03 | Verify indirect dependencies | Unit | Jest | [src/test/indirect_dependencies.test.ts](../src/test/indirect_dependencies.test.ts) |
|
||||
| TC-CONF-01 | schema_loader uses Config variables | Unit | Jest | [src/test/schema_config.test.ts](../src/test/schema_config.test.ts) |
|
||||
| TC-CONF-02 | constants are derived from Config | Unit | Jest | [src/test/zabbix_api_config.test.ts](../src/test/zabbix_api_config.test.ts) |
|
||||
| TC-CONF-03 | logger levels initialized from Config | Unit | Jest | [src/test/logger_config.test.ts](../src/test/logger_config.test.ts) |
|
||||
|
|
|
|||
69
docs/use-cases/trade-fair-logistics-requirements.md
Normal file
69
docs/use-cases/trade-fair-logistics-requirements.md
Normal file
|
|
@ -0,0 +1,69 @@
|
|||
# 🏗️ Trade Fair Logistics Requirements
|
||||
|
||||
This document outlines the requirements for extending the Zabbix GraphQL API to support trade fair logistics, derived from the analysis of the "KI-gestützte Orchestrierung in der Messelogistik" (AI-supported orchestration in trade fair logistics) pilot at Koelnmesse.
|
||||
|
||||
## 📋 Project Context
|
||||
The goal is to use the **Virtual Control Room (VCR)** as an orchestration platform to improve punctuality, throughput, and exception handling in trade fair logistics.
|
||||
|
||||
## 🛠️ Key Use Cases
|
||||
|
||||
- **Slot Risk Scoring & Proactive Rescheduling**:
|
||||
- *Description*: Proactive detection of missed delivery slots using ETAs and historical data.
|
||||
- *AI Function*: Calculates slot-miss risk and suggests next best actions (e.g. shift slot, alternative gate).
|
||||
- *Zabbix Role*: Monitoring ETA vs. Slot time, triggering alerts on high risk.
|
||||
|
||||
- **Exception Copilot for Dispatch & Gate**:
|
||||
- *Description*: Standardized workflows (Playbooks) for managing arrival deviations (e.g. no slot, wrong gate, missing documents).
|
||||
- *AI Function*: Classifies exceptions and provides communication templates.
|
||||
- *Zabbix Role*: Capturing exception events as triggers and managing the resolution state.
|
||||
|
||||
- **Multilingual Driver Assistant**:
|
||||
- *Description*: Step-by-step instructions for drivers on-site to reduce misunderstandings and wrong turns.
|
||||
- *Zabbix Role*: Providing real-time status updates (e.g. "Gate 4 is ready for you") to external communication interfaces.
|
||||
|
||||
- **Handling Readiness (Stapler/Personal/Rampe)**:
|
||||
- *Description*: Coordinating truck arrivals with the availability of handling resources like forklifts and ramps.
|
||||
- *Zabbix Role*: Monitoring the status and capacity of logistics assets and personnel.
|
||||
|
||||
- **VCR Setup Copilot**:
|
||||
- *Description*: Template-based configuration to scale the VCR for different venues (e.g. Koelnmesse, Düsseldorf) and varying event rules.
|
||||
- *Zabbix Role*: Management of venue-specific and event-specific host groups and templates.
|
||||
|
||||
## ⚙️ Technical Requirements for the API
|
||||
|
||||
- **Dynamic Device Modeling**:
|
||||
- Support for complex **Delivery** entities as Zabbix hosts.
|
||||
- Inclusion of dynamic attributes such as Slot-ID, ETA, and Gate assignments.
|
||||
|
||||
- **Hierarchical Data Mapping**:
|
||||
- Mapping nested logistics data (e.g. cargo details, handling status) to hierarchical Zabbix item structures.
|
||||
- Use of tags for classification and filtering of logistics tasks.
|
||||
|
||||
- **Real-time Telemetry Integration**:
|
||||
- High-frequency ingestion of GPS and sensor data (e.g. temperature, shock) from mobile tracking devices.
|
||||
- Support for Zabbix trapper items to receive external push updates.
|
||||
|
||||
- **AI-Integration Hooks**:
|
||||
- Enable external AI systems to push "Risk Scores" and "Next Best Actions" into Zabbix.
|
||||
- Use of Zabbix triggers to orchestrate AI-driven suggestions.
|
||||
|
||||
- **Workflow Orchestration**:
|
||||
- Ability to trigger external actions (e.g. sending notifications to drivers, creating tickets) based on Zabbix triggers.
|
||||
- Integration with the Model Context Protocol (MCP) to allow AI agents to manage logistics exceptions.
|
||||
|
||||
- **Multi-Venue Templates**:
|
||||
- Provision of reusable template structures for different exhibition centers and recurring events.
|
||||
- Support for bulk import/export of venue-specific configurations.
|
||||
|
||||
## ✅ KPIs for Success Measurement
|
||||
|
||||
- **Slot Hit Rate**: Percentage of vehicles arriving within their booked time window.
|
||||
- **P22-Quote**: Frequency of vehicles needing to be redirected to waiting areas (P22).
|
||||
- **Gate Waiting Time**: Average time from arrival at the venue to successful check-in.
|
||||
- **Throughput**: Number of vehicles processed per gate/hour.
|
||||
- **Average Handle Time (AHT)**: Mean time to resolve a logistics exception/ticket.
|
||||
- **First Contact Resolution**: Rate of exceptions resolved without further escalation.
|
||||
|
||||
## 🔗 References
|
||||
- **Analysis Document**: [docs/KI für Event- und Messelogistik.pdf](../KI%20f%C3%BCr%20Event-%20und%20Messelogistik.pdf)
|
||||
- **Roadmap**: [roadmap.md](../../roadmap.md)
|
||||
Loading…
Add table
Add a link
Reference in a new issue