misk

[Edit section]

[Copy link]

Misk is a microservice framework designed to simplify the development and deployment of microservices. It provides a set of tools and abstractions for building scalable and maintainable services, addressing common challenges in microservice architecture such as configuration management, database integration, networking, and cloud service integration.

The core of Misk is built around a modular architecture, with key components including:

1. Web Framework: Misk provides a robust web framework for handling HTTP requests and WebSocket connections. The framework is centered around the concept of Web Actions, which are defined in the misk-actions directory. Web Actions are the primary way to define endpoints in a Misk application. The framework handles request routing, parameter parsing, and response generation. Interceptors, defined in …/interceptors, allow for cross-cutting concerns such as authentication and logging to be applied across multiple actions.

2. Database Integration: Misk offers integration with various databases, primarily through Hibernate ORM and JDBC. The misk-hibernate directory contains the core functionality for Hibernate integration, including entity management, transactions, and custom types. JDBC integration is handled in the misk-jdbc directory, providing connection management and transaction support.

3. Configuration Management: The misk-config directory houses the configuration management system. It handles loading and parsing of configuration files, typically in YAML format, and provides a mechanism for overriding configuration values. The system also supports handling of secrets, allowing sensitive information to be stored securely.

4. Service Management: Misk includes a service lifecycle management system, implemented in the misk-service directory. This system coordinates the startup and shutdown of services, manages dependencies between services, and provides a framework for building a service graph.

5. Metrics and Monitoring: The misk-metrics directory contains the metrics system, which allows services to emit various types of metrics (counters, gauges, histograms) that can be collected and analyzed by monitoring systems.

6. Cloud Service Integration: Misk provides integration with various cloud services, including AWS (misk-aws), GCP (misk-gcp), and Redis (misk-redis). These integrations simplify the use of cloud services within Misk applications.

7. Admin Dashboard: Misk includes a web-based admin dashboard (misk-admin) for service administration and monitoring. The dashboard provides visibility into service metrics, configuration, and other runtime information.

The framework uses dependency injection extensively, primarily through Guice, to manage component lifecycles and dependencies. This approach allows for loose coupling between components and facilitates testing and configuration.

Misk's design emphasizes modularity and extensibility. Each major feature is encapsulated in its own module, allowing developers to include only the functionality they need. The framework also provides a comprehensive testing framework (misk-testing) to support unit and integration testing of Misk applications.

For more detailed information on specific components, refer to the corresponding sections in this wiki, such as Web Framework, Database Integration, or Configuration Management.

Web Framework

[Edit section]

[Copy link]

References: misk-actions, misk-api

The WebAction interface serves as the foundation for defining web actions in Misk. Web actions are registered using WebActionEntry, which allows customization of URL path prefixes. The WebActionModule handles the registration of these actions within the Guice dependency injection framework.

HTTP Request Handling

[Edit section]

[Copy link]

References: misk-api/src/main/kotlin/misk/web, misk-actions/src/main/kotlin/misk/web

• • •

HTTP request handling in Misk is primarily managed through annotations and interfaces defined in …/Http.kt. The framework uses these annotations to define endpoints, their HTTP methods, and parameter bindings:

WebSocket Support

[Edit section]

[Copy link]

References: misk-actions/src/main/kotlin/misk/web/actions

• • •

The WebSocket interface defines the core functionality for WebSocket connections, including methods for sending messages, closing connections, and managing the message queue. Key methods include:

Web Actions

[Edit section]

[Copy link]

References: misk-actions/src/main/kotlin/misk/web/actions

• • •

Web actions in Misk are defined using the WebAction interface, which serves as a marker interface for identifying classes that represent web actions. The WebActionEntry data class is used to register web actions with optional configuration, including the action class and URL path prefix.

Interceptors and Middleware

[Edit section]

[Copy link]

References: misk-actions/src/main/kotlin/misk/web/interceptors, misk-actions/src/main/kotlin/misk/security/authz

• • •

Misk provides interceptors and middleware for handling cross-cutting concerns in web applications. The LogRequestResponse annotation, defined in …/LogRequestResponse.kt, enables configurable logging of request and response information. Key features include:

Client-Side Networking

[Edit section]

[Copy link]

References: misk-api/src/main/kotlin/misk/client

• • •

The ClientNetworkChain interface defines the core structure for client-side network communication. It exposes two key properties:

Admin Dashboard Integration

[Edit section]

[Copy link]

References: misk-admin/src/main/kotlin/misk/web/metadata/guice

• • •

The Misk admin dashboard integrates with backend services through a series of Kotlin classes that manage metadata and facilitate source code navigation. The …/guice directory contains the GuiceDashboardTabModule, GuiceTabIndexAction, and GuiceSourceUrlProvider interface, along with its implementation GitHubSourceUrlProvider.

Metadata Management

[Edit section]

[Copy link]

References: misk-config/src/main/kotlin/misk/web/metadata

• • •

The Metadata class in …/Metadata.kt represents application metadata for the admin dashboard. It contains:

Database Integration

[Edit section]

[Copy link]

References: misk-hibernate, misk-jdbc

• • •

The Misk framework provides robust database integration through the misk-hibernate and misk-jdbc modules. These modules offer functionality for working with Hibernate ORM and JDBC respectively.

JDBC Testing Utilities

[Edit section]

[Copy link]

References: misk-jdbc/src/testFixtures/kotlin/misk/jdbc

• • •

In the Misk framework, the …/jdbc directory provides tools for establishing a consistent testing environment for JDBC integrations. These utilities are crucial for preparing a clean database state prior to each test, ensuring test reliability and isolation.

JDBC Configuration and Dependencies

[Edit section]

[Copy link]

References: misk-jdbc/build.gradle.kts

• • •

Integrating JDBC into Misk applications is streamlined through the …/build.gradle.kts script, which manages the necessary dependencies and build configurations. This script is pivotal for setting up JDBC-related functionality within the Misk framework, ensuring that applications can interact with databases using Java Database Connectivity.

Configuration Management

[Edit section]

[Copy link]

References: misk-config

• • •

The MiskConfig object in …/config is the central component for loading and managing configuration files. It supports:

Service Management

[Edit section]

[Copy link]

References: misk-service

• • •

The CoordinatedService class manages the lifecycle of interdependent services. It maintains sets of direct dependencies and all dependencies, coordinating startup and shutdown based on the state of these dependencies. The startIfReady() and stopIfReady() functions handle service state transitions, while findCycle() detects dependency cycles to ensure a valid directed acyclic graph.

Service Graph Management

[Edit section]

[Copy link]

References: misk-service/src/main/kotlin/misk/ServiceGraphBuilder.kt, misk-service/src/main/kotlin/misk/metadata/servicegraph/ServiceGraphMetadata.kt

• • •

The ServiceGraphBuilder class in …/ServiceGraphBuilder.kt constructs a graph of CoordinatedService instances to manage service startup and shutdown. It maintains:

Coordinated Services

[Edit section]

[Copy link]

References: misk-service/src/main/kotlin/misk/CoordinatedService.kt

The CoordinatedService class manages the lifecycle and dependencies of services within the Misk framework. It wraps a Service provider and maintains two sets of services:

Service Manager Module

[Edit section]

[Copy link]

References: misk-service/src/main/kotlin/misk/ServiceManagerModule.kt

• • •

The ServiceManagerModule configures and binds service-related components in Misk applications. It sets up a ServiceManager instance using provideServiceManager(), which takes a ServiceGraphBuilder and ServiceManager.Listener instances as input.

Metrics and Monitoring

[Edit section]

[Copy link]

References: misk-metrics

• • •

The Metrics interface in …/Metrics.kt serves as the primary entry point for application code to interact with the metrics system. It provides methods for creating and registering various metric types:

Metrics Infrastructure

[Edit section]

[Copy link]

References: misk-metrics

• • •

In the Misk framework, the metrics collection and reporting infrastructure is centered around the CollectorRegistry, which serves as the central store for all metric collectors. The setup of the CollectorRegistry is crucial for the aggregation and retrieval of metrics data within an application. The integration with the application's dependency injection system is managed through the CollectorRegistryModule, which is defined in …/CollectorRegistryModule.kt. This module binds the CollectorRegistry to the dependency injection system, ensuring that it is available for use throughout the application.

Testing Metrics

[Edit section]

[Copy link]

References: misk-metrics/src/testFixtures/kotlin/misk/metrics/v2

• • •

In the Misk application, the testing of metrics collection is facilitated by utilities and fixtures located in the …/v2 directory. A key component in this setup is the CollectorRegistryFixture, which serves as a central point for managing the state of metrics during test runs. This fixture extends the functionality of the CollectorRegistry by implementing the TestFixture interface, which is crucial for ensuring a consistent testing environment.

Cloud Service Integration

[Edit section]

[Copy link]

References: misk-aws, misk-gcp, misk-redis

The Misk framework provides integration with various cloud services through dedicated modules:

Redis Integration

[Edit section]

[Copy link]

References: misk-redis

• • •

Integrating Redis into Misk applications is facilitated through the misk-redis directory, which houses the necessary classes and configurations for Redis client interactions and lifecycle management. The Redis interface, located at …/Redis.kt, serves as the primary API for Redis operations, offering methods like get(), set(), and del(). Implementations of this interface include RealRedis, which utilizes UnifiedJedis for actual Redis interactions, and FakeRedis for testing with a simulated Redis environment.

Redis Configuration

[Edit section]

[Copy link]

References: misk-redis/src/main/kotlin/misk/redis

• • •

Configuring Redis clients within Misk applications involves setting up replication groups and managing cluster connections, primarily handled by RedisClusterConfig. This configuration class is crucial for specifying the necessary parameters for connecting to a Redis cluster, including host addresses and ports, as well as any authentication requirements.

Redis Testing Support

[Edit section]

[Copy link]

References: misk-redis/src/test/kotlin/misk/redis, misk-redis/src/testFixtures/kotlin/misk/redis/testing

• • •

Testing Redis-dependent components in Misk applications is facilitated by a suite of utilities and configurations that simulate real-world scenarios. The …/redis directory contains test suites for various Redis client behaviors, including pipelined operations, transactions, key scanning, and connection pool metrics. These tests ensure that the Redis client implementation within Misk functions correctly under different conditions.

Cryptography

[Edit section]

[Copy link]

References: misk-crypto

• • •

The CryptoModule class configures and registers cryptographic keys and providers in a Guice-based dependency injection system. It reads the CryptoConfig object and sets up bindings for various cryptographic primitives.

Feature Flags

[Edit section]

[Copy link]

References: misk-feature

• • •

The FeatureFlags interface in …/FeatureFlags.kt provides methods for evaluating and tracking various types of feature flags:

Feature Flag Testing Utilities

[Edit section]

[Copy link]

References: misk-feature/src/testFixtures/kotlin/misk/feature/testing

• • •

In the Misk framework, feature flag testing is facilitated by the FakeFeatureFlags class, which serves as a mock implementation of the FeatureFlags interface. Located in …/FakeFeatureFlags.kt, this class allows developers to simulate various states of feature flags within a testing environment by maintaining a map of feature flag overrides. These overrides can be manipulated to test different configurations and behaviors driven by feature flags, without the need to interact with the actual feature flagging system.

Feature Flag Management

[Edit section]

[Copy link]

References: misk-feature/build.gradle.kts

Feature flags in Misk are managed through the build configuration in …/build.gradle.kts, which sets up the necessary dependencies and plugins to support feature flag functionality within the application. The management of feature flags involves the following key aspects:

Event Processing

[Edit section]

[Copy link]

References: misk-events

• • •

The Misk event system provides functionality for publishing and consuming events, with support for local spooling and in-memory testing.

Admin Dashboard

[Edit section]

[Copy link]

References: misk-admin

• • •

The Misk Admin Dashboard provides a web-based interface for service administration and monitoring. The dashboard is built using the following key components:

Dashboard Modules

[Edit section]

[Copy link]

References: misk-admin/src/main/kotlin/misk/web/dashboard

• • •

The Misk admin dashboard is architected to offer a centralized web interface for service management and monitoring, facilitated by a collection of modules that define its structure and capabilities. A notable addition to the dashboard is the GuiceDashboardTabModule, which enhances the dashboard's introspective features by visualizing Guice bindings and metadata, essential for understanding the dependency injection setup within a Misk application.

Guice Bindings Metadata Interface

[Edit section]

[Copy link]

References: misk-admin/src/main/kotlin/misk/web/metadata/guice/GuiceTabIndexAction.kt

• • •

The GuiceTabIndexAction class serves as the backbone for a dashboard page dedicated to visualizing the Guice dependency injection bindings within a Misk application. It leverages the DashboardPageLayout to structure the page, ensuring a consistent layout and design across the admin dashboard.

Service Graph Visualization

[Edit section]

[Copy link]

References: misk-admin/src/main/kotlin/misk/web/metadata/servicegraph/ServiceGraphTabIndexAction.kt

• • •

The ServiceGraphTabIndexAction class renders the service graph visualization on the admin dashboard. It utilizes D3.js for creating an interactive graph representation of the service dependencies. The visualization process involves:

Dashboard Static Web Assets

[Edit section]

[Copy link]

References: misk-admin/src/main/resources/web/static/controllers/guice_search_bar_controller.js

• • •

The Misk Admin Dashboard leverages the …/controllers directory to enhance user interaction through Stimulus controllers. These controllers add dynamic behaviors to the dashboard's web pages, enabling features like real-time content filtering.

Core Utilities

[Edit section]

[Copy link]

References: misk-core, wisp

• • •

The misk-core module provides essential utilities and abstractions used throughout the Misk framework. Key components include:

Time Utilities

[Edit section]

[Copy link]

References: wisp/wisp-time-testing

The …/wisp-time-testing directory is dedicated to providing utilities for manipulating and controlling time within test environments. Central to this is the FakeClock class, which simulates the progression of time, essential for testing time-dependent functionalities without reliance on the system clock.

Dependency Injection

[Edit section]

[Copy link]

References: misk-inject

• • •

The Misk framework leverages the Guice dependency injection framework to manage object creation and resolve dependencies, offering a Kotlin-friendly interface for developers. The misk-inject directory is central to this functionality, housing classes that extend and enhance Guice's capabilities to fit the idiomatic style of Kotlin.

Retry Configuration and Execution

[Edit section]

[Copy link]

References: misk-core/src/main/kotlin/misk/backoff

• • •

In Misk, the retry logic is facilitated by the retry function, which is designed to execute a block of code multiple times with a specified backoff strategy in case of failure. The function is located in …/Retries.kt. The retry function accepts a lambda and executes it, allowing for retries up to a maximum number specified by the upTo parameter. If the lambda throws an exception, the retry function applies a backoff strategy provided by the withBackoff parameter before attempting to execute the lambda again.

Testing Framework

[Edit section]

[Copy link]

References: misk-testing

• • •

The Misk testing framework provides utilities and abstractions for testing Misk applications. Key components include:

Injector Reuse in Testing

[Edit section]

[Copy link]

References: misk-testing/README.md, misk-testing/src/main/kotlin/misk/testing/MiskTestExtension.kt

• • •

In Misk, the experimental feature for injector reuse aims to enhance test performance by maintaining a single instance of the Injector and the service lifecycle across multiple tests. This approach minimizes the overhead associated with setting up and tearing down the test environment for each individual test, which can be particularly beneficial in large test suites or when the initialization process is resource-intensive.

Test Fixtures and Resettable Properties

[Edit section]

[Copy link]

References: misk-testing-api/src/main/kotlin/misk/testing/TestFixture.kt

• • •

In the context of testing Misk applications, the TestFixture interface plays a crucial role in managing the lifecycle of test fixtures. Implemented by test fixtures, the reset() method is essential for ensuring that the state of these fixtures is returned to a predefined state before each test run. This is particularly important in a microservice environment where tests may need to run in isolation and without side effects from previous tests.

Testing Dependencies

[Edit section]

[Copy link]

References: misk-testing/build.gradle.kts, misk-testing-api/build.gradle.kts

In the Misk framework, the addition and configuration of new testing dependencies are managed within the build files located at …/build.gradle.kts and …/build.gradle.kts. These files are crucial for setting up the testing environment, ensuring that the necessary libraries and tools are available for developers to write and run tests effectively.

Testing Service Configuration

[Edit section]

[Copy link]

References: misk-testing/src/main/kotlin/misk/MiskTestingServiceModule.kt

• • •

The MiskTestingServiceModule serves as a critical component for configuring the testing environment in Misk applications. It replaces real-world dependencies with fakes, enabling isolated and controlled unit testing. The module is particularly useful for simulating external resources like environment variables and filesystem dependencies, which are often difficult to manage in a test setting.