WTX

WTX is a human-written collection of different transport implementations and related tools focused primarily on web technologies. Features the in-house development of +10 IETF RFCs along side other elements.

Works on embedded devices with heap allocators. If you find this crate interesting, please consider giving it a star ⭐ on GitHub.

High-level benchmarks

Checkout wtx-bench or HttpArena and feel free to indicate possible misunderstandings.

gRPC	HTTP/2	WebSocket

Low-level benchmarks

Anything marked with #[bench] in the repository is considered a low-level benchmark in the sense that they measure very specific operations that generally serve as the basis for other parts.

Take a look at https://bencher.dev/perf/wtx to see all low-level benchmarks over different periods of time.

Development benchmarks

These numbers provide an estimate of the expected waiting times when developing with WTX. If desired, you can compare them with other similar Rust projects through the dev-bench.sh script.

Crypto Backend

Taking aside very few exceptions, WTX does not have built-in cryptographic algorithms, as such, it is necessary to select a backend when working with features that require them.

• crypto-aws-lc-rs
• crypto-graviola
• crypto-openssl
• crypto-ring

Calling methods will halt/panic the application if no backend is selected. These panicking branches will hopefully be erased by dead code analysis if the crypto feature is somehow active but never actually used.

In practice many things require cryptography algorithms. For example, PostgreSQL uses HMAC and secure HTTP cookie uses AEAD.

Examples

Demonstrations of different use-cases can be found in the wtx-examples directory as well as in the documentation located at https://c410-f3r.github.io/wtx.

Limitations

• WTX is not widely used and has not undergone security audits, therefore, use it at your own risk. However, reproducible contributions that improve security are always welcome.

• Does not support systems with a pointer length of 16 bits.

• Expects the infallible sum of the lengths of an arbitrary number of slices, otherwise the program will likely trigger an overflow that can possibly result in unexpected operations. For example, in a 32bit system such a scenario should be viable without swap memory or through specific limiters like ulimit.

1. Internal dependencies required by the feature. ↩

2. The sum of optional and required dependencies used by the associated binaries. ↩

Calendar

Provides basic primitives to work with time-related operations.

• Date: Proleptic Gregorian calendar. Can represent years from -32767 to 32767.

• DateTime: ISO-8601 representation with timezones.

• Duration: Time span in nanoseconds. Can be negative unlike core::time::Duration.

• Instant: A specific point in time. Contains the underlying mechanism that provides a timestamp.

• Time Clock time with nanosecond precision.

Also supports arithmetic operations and flexible formatting.

Embedded devices

no_std users that utilize the embassy crate should first make a UDP request to a NTP server and then call the wtx::calendar::set_epoch_offset function once at start-up, otherwise timestamps will represent the time since boot.

Example

//! Basic time operation.

extern crate wtx;

use wtx::calendar::{Duration, Instant};

fn main() -> wtx::Result<()> {
  println!(
    "ISO 8601 representation of the next 2 minutes in UTC: {}",
    Instant::now_date_time()?.add(Duration::from_minutes(2)?)?
  );
  Ok(())
}

Client API Framework

A flexible client API framework for writing asynchronous, fast, organizable, scalable and maintainable applications. Supports several data formats, transports and custom parameters.

Checkout the wtx-apis project to see a collection of APIs based on wtx.

To use this functionality, it is necessary to activate the client-api-framework feature.

Objective

It is possible to directly decode responses using built-in methods provided by transport implementations like reqwest but as complexity grows, the cost of maintaining large sets of endpoints with ad-hoc solutions usually becomes unsustainable. Based on this scenario, wtx comes into play to organize and centralize data flow in a well-defined manner to increase productivity and maintainability.

For API consumers, the calling convention of wtx endpoints is based on fluent interfaces which makes the usage more pleasant and intuitive.

Moreover, the project may in the future create automatic bindings for other languages in order to avoid having duplicated API repositories.

Database Client

Provides a set of functions that establish connections, execute queries and manage data transactions.

Independent benchmarks are available at https://github.com/diesel-rs/metrics.

PostgreSQL

Implements a subset of https://www.postgresql.org/docs/16/protocol.html. PostgreSQL is a robust, open-source relational database management system that, among other things, supports several data types and usually also excels in concurrent scenarios.

To use this functionality, it is necessary to activate the postgres feature.

Prepared statements

//! Demonstrates different interactions with a PostgreSQL database.

extern crate tokio;
extern crate wtx;
extern crate wtx_examples;

use wtx::{
  database::{DbClient as _, Record, Records},
  misc::into_rslt,
};

#[tokio::main]
async fn main() -> wtx::Result<()> {
  let uri = "postgres://USER:PASSWORD@localhost/DATABASE";
  let mut executor = wtx_examples::postgres_client(uri).await?;
  executor
    .transaction(|this| async {
      this.execute_ignored("CREATE TABLE IF NOT EXISTS example(id INT, name VARCHAR)").await?;
      this
        .execute_stmt_none("INSERT INTO foo VALUES ($1, $2), ($3, $4)", (1u32, "one", 2u32, "two"))
        .await?;
      Ok(((), this))
    })
    .await?;
  let records = executor
    .execute_stmt_many("SELECT id, name FROM example", (), |_| Ok::<_, wtx::Error>(()))
    .await?;
  let record0 = into_rslt(records.get(0))?;
  let record1 = into_rslt(records.get(1))?;
  assert_eq!((record0.decode::<_, u32>(0)?, record0.decode::<_, &str>("name")?), (1, "one"));
  assert_eq!((record1.decode::<_, u32>("id")?, record1.decode::<_, &str>(1)?), (2, "two"));
  Ok(())
}

Batch

PostgreSQL supports the sending of multiple statements in a single round-trip.

//! Sends multiple commands at once and awaits them.

extern crate tokio;
extern crate wtx;
extern crate wtx_examples;

use wtx::{
  collections::ArrayVectorU8,
  database::{Record, Records},
  misc::into_rslt,
};

const COMMANDS: &[&str] = &["SELECT 0 = $1", "SELECT 1 = $1", "SELECT 2 = $1"];

#[tokio::main]
async fn main() -> wtx::Result<()> {
  let uri = "postgres://USER:PASSWORD@localhost/DATABASE";
  let mut executor = wtx_examples::postgres_client(uri).await?;
  let mut batch = executor.batch();
  let mut idx: u32 = 0;
  let mut records = ArrayVectorU8::<_, { COMMANDS.len() }>::new();
  for cmd in COMMANDS {
    batch.stmt(cmd, (idx,))?;
    idx = idx.wrapping_add(1);
  }
  batch.flush(&mut records, |_| Ok(())).await?;
  for record in records {
    assert!(into_rslt(record.get(0))?.decode::<_, bool>(0)?);
  }
  Ok(())
}

Tests

The #[wtx::db] macro automatically migrates and seeds individual tests in isolation to allow concurrent evaluations. Connected users must have the right to create new databases.

To use this functionality, it is necessary to activate the database-tests feature.

//! The DB isolation provided by `#[wtx::db]` allows the concurrent execution of identical queries
//! without causing conflicts.
//!
//! The `dir` parameter is optional.

fn main() {}

#[cfg(test)]
mod tests {
  use tokio::net::TcpStream;
  use wtx::database::{
    DbClient, Record,
    client::postgres::{ClientBuffer, PostgresClient},
  };

  type LocalPostgresClient = PostgresClient<ClientBuffer, wtx::Error, TcpStream>;

  #[wtx::db(dir("../.test-utils"))]
  async fn first_test(client: LocalPostgresClient) {
    common(client).await;
  }

  #[wtx::db(dir("../.test-utils"))]
  async fn second_test(client: LocalPostgresClient) {
    common(client).await;
  }

  #[wtx::db(dir("../.test-utils"))]
  async fn third_test(client: LocalPostgresClient) {
    common(client).await;
  }

  async fn common(mut client: LocalPostgresClient) {
    client
      .execute_ignored(
        "
          CREATE TABLE foo(id INT PRIMARY KEY, description TEXT NOT NULL);
          INSERT INTO foo VALUES (1, 'BAR!');
        ",
      )
      .await
      .unwrap();
    let id: &str = client.execute_single("SELECT * FROM foo").await.unwrap().decode(0).unwrap();
    assert_eq!(id, "1");
  }
}

Encrypted Connections

It usually isn’t straightforward to stablish encrypted connections with PostgreSQL, worse yet, wtx has a set of limited SSL policies that doesn’t allow the by-passing of invalid certificates.

The following sections will briefly demonstrate how to configure both servers and clients to establish encrypted connections using self-signed certificates with Podman or Docker. Most of the procedures can be adapted for non-containerized environments and also for certificates issued by trusted actors.

In case of doubt, always remember that a server needs a key and a certificate while both parties need a root authority certificate. Sometimes even a CA certificate isn’t necessary.

Generate certificates

Just an example, you can use other tools like cert-manager or other algorithms like ed25519.

CERTS_DIR="SOME_DIRECTORY"
openssl req -newkey rsa:2048 -nodes -subj "/C=FI/CN=vahid" -keyout $CERTS_DIR/key.pem -out $CERTS_DIR/key.csr
openssl x509 -signkey $CERTS_DIR/key.pem -in $CERTS_DIR/key.csr -req -days 1825 -out $CERTS_DIR/cert.pem
openssl req -x509 -sha256 -nodes -subj "/C=FI/CN=vahid" -days 1825 -newkey rsa:2048 -keyout $CERTS_DIR/root-ca.key -out $CERTS_DIR/root-ca.crt
cat <<'EOF' >> $CERTS_DIR/localhost.ext
authorityKeyIdentifier=keyid,issuer
basicConstraints=CA:FALSE
subjectAltName = @alt_names
[alt_names]
DNS.1 = localhost
IP.1 = 127.0.0.1
EOF
openssl x509 -req -CA $CERTS_DIR/root-ca.crt -CAkey $CERTS_DIR/root-ca.key -in $CERTS_DIR/key.csr -out $CERTS_DIR/cert.pem -days 1825 -CAcreateserial -extfile $CERTS_DIR/localhost.ext
rm $CERTS_DIR/key.csr
rm $CERTS_DIR/localhost.ext
rm $CERTS_DIR/root-ca.srl

PostgreSQL

You need to place these certificates in the container at the specified location AND set the same files as read-only for the current user. As far as I can tell, there are three possible ways.

1. Create a custom Docker image.
2. List a set of volume mappings alongside some initial script.
3. Inline certificates in docker-entrypoint-initdb.d

Let’s use option 3 for the sake of simplicity with a script named setup.sh.

#!/usr/bin/env bash

echo "Contents of the generated root CA certificate file" > $PGDATA/root-ca.crt
echo "Contents of the generated certificate file" > $PGDATA/cert.pem
echo "Contents of the generated key file" > $PGDATA/cert.pem

chmod 0600 $PGDATA/cert.pem $PGDATA/key.pem

cat >> "$PGDATA/postgresql.conf" <<-EOF
ssl = on
ssl_ca_file = 'root-ca.crt'
ssl_cert_file = 'cert.pem'
ssl_key_file = 'key.pem'
EOF

Everything should be ready on the server side.

podman run \
  --name SOME_CONTAINER_NAME \
  -d \
  -e POSTGRES_DB=SOME_DB \
  -e POSTGRES_PASSWORD=SOME_PASSWORD \
  -p 5432:5432 \
  -v SOME_DIRECTORY/setup.sh:/docker-entrypoint-initdb.d/setup.sh \
  docker.io/library/postgres:18

Now it is just a matter of including the root CA certificate in the wtx client. With everything properly configured, a successful encrypted connection should be expected.

#![allow(unused)]
fn main() {
extern crate wtx;

pub async fn postgres_client(
  root_ca: &[u8],
  uri_str: &str,
) -> wtx::Result<wtx::database::client::postgres::PostgresClient<
  wtx::Error,
  std::net::TcpStream,
  wtx::tls::TlsModeVerified
>> {
  use std::net::TcpStream;
  use wtx::{
    database::client::postgres::{ClientBuffer, Config, PostgresClient},
    rng::{ChaCha20, CryptoSeedableRng},
    tls::{TlsConfig, TlsConnector},
  };
  let uri = wtx::misc::Uri::new(uri_str);
  let mut tls_connector = TlsConnector::new(
    TlsConfig::from_trust_anchors_pem(wtx::tls::TlsModeVerified::default(), [root_ca])?,
    ChaCha20::from_getrandom()?,
    TcpStream::connect(uri.hostname_with_implied_port())?,
  );
  PostgresClient::connect(
    ClientBuffer::new(usize::MAX, tls_connector.rng_mut()),
    &Config::from_uri(&uri)?,
    tls_connector,
  )
  .await
}
}

Database Schema Management

Embedded and CLI workflows using raw SQL commands. A schema manager is a tool thats allows developers to define, track and apply changes to database structures over time, ensuring consistency across different environments.

To use this functionality, it is necessary to activate the schema-manager feature.

CLI

# Example

cargo install --git https://github.com/c410-f3r/wtx --features schema-manager-dev wtx-ui
echo DATABASE_URI="postgres://USER:PASSWORD@localhost:5432/DATABASE" > .env
RUST_LOG=debug wtx-cli migrate

The CLI application expects a configuration file that contains a set of paths where each path is a directory with multiple migrations.

# wtx.toml

migration_groups = [
  "migrations/1__initial",
  "migrations/2__fancy_stuff"
]

Each provided migration and group must contain an unique version and a name summarized by the following structure:

// Execution order of migrations is dictated by their numeric declaration order.

migrations
+-- 1__initial (Group)
    +-- 1__create_author.sql (Migration)
    +-- 2__create_post.sql (Migration)
+-- 2__fancy_stuff (Group)
    +-- 1__something_fancy.sql (Migration)
wtx.toml

The SQL file itself is composed by two parts, one for migrations (-- wtx IN section) and another for rollbacks (-- wtx OUT section).

-- wtx IN

CREATE TABLE author (
  id INT NOT NULL PRIMARY KEY,
  added TIMESTAMPTZ NOT NULL,
  birthdate DATE NOT NULL,
  email VARCHAR(100) NOT NULL,
  first_name VARCHAR(50) NOT NULL,
  last_name VARCHAR(50) NOT NULL
);

-- wtx OUT

DROP TABLE author;

One cool thing about the expected file configuration is that it can also be divided into smaller pieces, for example, the above migration could be transformed into 1__author_up.sql and 1__author_down.sql.

-- 1__author_up.sql

CREATE TABLE author (
  id INT NOT NULL PRIMARY KEY,
  added TIMESTAMPTZ NOT NULL,
  birthdate DATE NOT NULL,
  email VARCHAR(100) NOT NULL,
  first_name VARCHAR(50) NOT NULL,
  last_name VARCHAR(50) NOT NULL
);

-- 1__author_down.sql

DROP TABLE author;

migrations
+-- 1__some_group (Group)
    +-- 1__author (Migration directory)
        +-- 1__author_down.sql (Down migration)
        +-- 1__author_up.sql (Up migration)
        +-- 1__author.toml (Optional configuration)
wtx.toml

Library

The library gives freedom to arrange groups and uses some external crates, bringing ~10 additional dependencies into your application. If this overhead is not acceptable, then you probably should discard the library and use the CLI binary instead as part of a custom deployment strategy.

extern crate wtx;

use std::path::Path;
use wtx::database::{schema_manager::Commands, DEFAULT_URI_VAR};
use wtx::collections::Vector;

#[wtx::main]
async fn main() {
  let mut commands = Commands::with_executor(());
  commands.migrate_from_dir(Path::new("my_custom_migration_group_path")).await.unwrap();
}

Embedded migrations

To make deployment easier, the final binary of your application can embed all necessary migrations through the binary that is available in the wtx-ui crate.

#![allow(unused)]
fn main() {
extern crate wtx;

// This is an example! The actual contents are filled by the `wtx-ui embed-migrations` binary call.
mod embedded_migrations {
  pub(crate) static GROUPS: wtx::database::schema_manager::EmbeddedMigrationsTy = &[];
}

use wtx::database::schema_manager::Commands;
use wtx::collections::Vector;

async fn migrate() -> wtx::Result<()> {
  Commands::with_executor(()).migrate_from_groups(embedded_migrations::GROUPS).await
}
}

Conditional migrations

If one particular migration needs to be executed in a specific set of databases, then it is possible to use the -- wtx dbs parameter in a file.

-- wtx dbs mssql,postgres

-- wtx IN

CREATE SCHEMA foo;

-- wtx OUT

DROP SCHEMA foo;

Repeatable migrations

Repeatability can be specified with -- wtx repeatability SOME_VALUE where SOME_VALUE can be either always (regardless of the checksum) or on-checksum-change (runs only when the checksums changes).

-- wtx dbs postgres
-- wtx repeatability always

-- wtx IN

CREATE OR REPLACE PROCEDURE something() LANGUAGE SQL AS $$ $$

-- wtx OUT

DROP PROCEDURE something();

Keep in mind that repeatable migrations might break subsequent operations, therefore, you must known what you are doing. If desirable, they can be separated into dedicated groups.

migrations/1__initial_repeatable_migrations
migrations/2__normal_migrations
migrations/3__final_repeatable_migrations

Namespaces/Schemas

For supported databases, there is no direct user parameter that inserts migrations inside a single database schema but it is possible to specify the schema inside the SQL file and arrange the migration groups structure in a way that most suits you.

-- wtx IN

CREATE TABLE cool_department_schema.author (
  id INT NOT NULL PRIMARY KEY,
  full_name VARCHAR(50) NOT NULL
);

-- wtx OUT

DROP TABLE cool_department_schema.author;

Environment Variables

The EnvVars structure allows the insertion of environment variables into a custom container where the name of the fields match the name of the variables. .env files are also supported but they should be restricted to development environments.

The unsafe std::env::set_var function is not invoked due to concerns about concurrent access, therefore, direct usage of std::env::var is not recommended unless:

1. EnvVars is not used at all.
2. There are no .env files.
3. A specific variable is always originated from the current process.

Example

//! `EnvVars` allows the interactive reading of environment variables.

extern crate wtx;

use std::sync::OnceLock;
use wtx::{
  calendar::{DateTime, Utc},
  collections::Vector,
  misc::EnvVars,
};

static VARS: OnceLock<Vars> = OnceLock::new();

fn main() -> wtx::Result<()> {
  let _rslt = VARS.set(EnvVars::from_available([])?.finish());
  let Vars { now, origin, port, root_ca, rust_log } = VARS.wait();
  println!(
    "`NOW={now:?}`, `ORIGIN={origin}`, `PORT={port}`, `ROOT_CA={root_ca:?}` and `RUST_LOG={rust_log:?}`"
  );
  Ok(())
}

#[derive(Debug, wtx::FromVars)]
struct Vars {
  #[from_vars(map_now)]
  now: Option<DateTime<Utc>>,
  origin: String,
  #[from_vars(map_port)]
  port: u16,
  root_ca: Vector<u8>,
  rust_log: Option<String>,
}

fn map_now(var: String) -> wtx::Result<DateTime<Utc>> {
  DateTime::from_iso8601(var.as_bytes())
}

fn map_port(var: String) -> wtx::Result<u16> {
  Ok(var.parse()?)
}

Error Handling

The majority of operations performed by WTX is fallible, in other words, most functions or methods return a Result enum instead of panicking under the hood. A considerable effort is put to hint the compiler that a branch is unreachable to optimize code generation but that is another topic.

Due to this characteristic downstream users are encouraged to create their own Error enum with a WTX variant alongside a From trait implementation. Not to mention the unlocking of the useful ? operator that performs the automatic conversion of any supported error element.

extern crate wtx;

use wtx::codec::FromRadix10;

#[derive(Debug)]
pub enum Error {
    MyDogAteMyHomework,
    RanOutOfCoffee,
    Wtx(wtx::Error)
}

impl From<wtx::Error> for Error {
    fn from(from: wtx::Error) -> Self {
        Self::Wtx(from)
    }
}

fn main() -> Result<(), Error> {
    let _u16_from_bytes = u16::from_radix_10(&[49][..])?;
    let _u16_from_i8 = u16::try_from(1i8).map_err(wtx::Error::from)?;
    Ok(())
}

All these conventions are of course optional. If desired everything can be unwrapped using the Result::unwrap method.

When you encounter an error, try take a look at the available documentation of that specific error (https://docs.rs/wtx/latest/wtx/enum.Error.html). If the documentation didn’t help, feel free to reach out for potential improvements.

Executor

Simple dependency-free runtime intended for tests, toy programs and demonstrations. Performance is not a main concern and you should probably use other executors like tokio.

To use this functionality, it is necessary to activate the executor feature.

Example

//! The `executor` feature allows the execution of asynchronous operations

extern crate wtx;

#[wtx::main]
async fn main() {
  println!("With great power comes great electricity bills");
}

#[wtx::test]
async fn test_with_runtime(runtime: &wtx::executor::StdRuntime) {
  runtime
    .spawn(async move {
      println!("Behind every successful diet is an unwatched pizza");
    })
    .unwrap();
}

gRPC

Basic implementation that currently only supports unary calls. gRPC is an high-performance remote procedure call framework developed by Google that enables efficient communication between distributed systems, particularly in microservices architectures.

wtx does not provide built-in deserialization or serialization utilities capable of manipulate protobuf files. Instead, users are free to choose any third-party that generates Rust bindings and implements the internal Deserialize and Serialize traits.

Due to the lack of an official parser, the definitions of a Service must be manually typed.

Independent benchmarks are available at https://github.com/LesnyRumcajs/grpc_bench.

Client Example

To use this functionality, it is necessary to activate the grpc-client feature.

//! gRPC client that uses the structure definitions found in the `wtx_instances::grpc_bindings`
//! module.

extern crate tokio;
extern crate wtx;
extern crate wtx_examples;

use std::borrow::Cow;
use wtx::{
  codec::format::QuickProtobuf,
  executor::TokioExecutor,
  grpc::GrpcClient,
  http::{MsgBufferStr, http2_client_pool::Http2ClientPoolBuilder},
  rng::{ChaCha20, CryptoSeedableRng as _},
  tls::{TlsConfig, TlsModeVerified},
};
use wtx_examples::{
  ROOT_CA,
  grpc_bindings::wtx::{GenericRequest, GenericResponse},
};

#[tokio::main]
async fn main() -> wtx::Result<()> {
  let uri = "https://127.0.0.1:9000/wtx.GenericService/generic_method";
  let mut client = GrpcClient::new(
    Http2ClientPoolBuilder::new(
      TokioExecutor::default(),
      1,
      ChaCha20::from_getrandom()?,
      TlsConfig::from_trust_anchors_pem(TlsModeVerified::default(), [ROOT_CA])?,
    )?
    .build(),
    QuickProtobuf,
  );
  let res = client
    .send_unary_req(
      GenericRequest {
        generic_request_field0: Cow::Borrowed(b"generic_request_value"),
        generic_request_field1: 123,
      },
      MsgBufferStr::from_uri(uri.into()),
    )
    .await?;
  let generic_response: GenericResponse = client.des_from_res_bytes(&res.msg_data.body)?;
  println!("{generic_response:?}");
  Ok(())
}

Server Example

To use this functionality, it is necessary to activate the grpc-server feature.

//! gRPC server that uses the structure definitions found in the `wtx_instances::grpc_bindings`
//! module.

extern crate tokio;
extern crate wtx;
extern crate wtx_examples;

use std::borrow::Cow;
use wtx::{
  codec::format::QuickProtobuf,
  grpc::{GrpcManager, GrpcMiddleware},
  http::{
    StatusCode,
    http2_server_framework::{Http2ServerFramework, HttpRouter, State, post},
  },
  tls::{TlsConfig, TlsModeVerified},
};
use wtx_examples::{
  PUBLIC_KEY, SECRET_KEY,
  grpc_bindings::wtx::{GenericRequest, GenericResponse},
  host_from_args,
};

fn main() -> wtx::Result<()> {
  let router = HttpRouter::new(
    wtx::paths!(("wtx.GenericService/generic_method", post(wtx_generic_service_generic_method))),
    GrpcMiddleware,
  )?;
  Http2ServerFramework::tokio(TlsConfig::from_keys_pem(
    TlsModeVerified::default(),
    PUBLIC_KEY.try_into()?,
    SECRET_KEY.try_into()?,
  )?)?
  .set_data(GrpcManager::from_drsr(QuickProtobuf))
  .run_in_threads(&host_from_args(), router)
}

async fn wtx_generic_service_generic_method(
  state: State<'_, GrpcManager<QuickProtobuf>>,
) -> wtx::Result<StatusCode> {
  let _generic_request: GenericRequest = state.data.des_from_req_bytes(&state.req.msg_data.body)?;
  state.req.clear();
  state.data.ser_to_res_bytes(
    &mut state.req.msg_data.body,
    GenericResponse {
      generic_response_field0: Cow::Borrowed(b"generic_response_value"),
      generic_response_field1: 321,
    },
  )?;
  Ok(StatusCode::Ok)
}

HTTP/2

Implementation of RFC7541 and RFC9113. HTTP/2 is the second major version of the Hypertext Transfer Protocol, introduced in 2015 to improve web performance, it addresses limitations of HTTP/1.1 while maintaining backwards compatibility.

Passes the hpack-test-case and the h2spec test suites. Due to official and unofficial deprecations, prioritization and server-push are not supported.

There are a bunch of low-level details that most individuals don’t care about when they are building applications. If that is your case, high level interfaces are available in http2-client-pool or http2-server-framework.

To use this functionality, it is necessary to activate the http2 feature.

HTTP/1.1 Upgrade

Does not support upgrading from HTTP/1.1 because browsers also don’t support such a feature. Connections must be established directly using HTTP/2 (h2c) or via ALPN (Application-Layer Protocol Negotiation) during the TLS handshake.

Operating Modes

There are two distinct operating modes for handling data transmission.

Automatic Mode

The system takes full responsibility. When you provide a buffer of data to be sent, the implementation automatically fragments it into appropriate DATA frames based on the maximum frame size and the current flow control window.

Manual Mode

Allows more control but you should know HTTP/2 concepts and their interactions. In this mode the user is responsible for constructing and sending individual HEADERS, DATA and TRAILERS frames.

Client Example

//! Fetches an URI using low-level HTTP/2 resources.

extern crate tokio;
extern crate wtx;

use tokio::net::TcpStream;
use wtx::{
  collections::Vector,
  http::{HttpClient, HttpRecvParams, ReqBuilder},
  http2::{Http2, Http2Buffer, Http2ErrorCode},
  misc::{Uri, from_utf8_basic},
  rng::{ChaCha20, CryptoSeedableRng},
  stream::Stream,
  tls::{TlsConfig, TlsConnector, TlsModeVerified},
};

#[tokio::main]
async fn main() -> wtx::Result<()> {
  let uri = Uri::new("https://github.com/c410-f3r/wtx");
  let stream = TcpStream::connect(uri.hostname_with_implied_port()).await?;
  let mut rng = ChaCha20::from_getrandom()?;
  let hb = Http2Buffer::new(&mut rng);
  let hrp = HttpRecvParams::with_optioned_params();
  let tls_config = TlsConfig::from_ccadb(TlsModeVerified::default())?;
  let tcr = TlsConnector::new(tls_config, rng, stream).connect().await?;
  let (frame_reader, http2) = Http2::connect(hb, hrp, tcr.tls_stream.into_split()?).await?;
  let _jh = tokio::spawn(frame_reader);
  let res = http2
    .send_req_recv_res(&mut Vector::new(), ReqBuilder::get(uri.to_ref()).into_request())
    .await?;
  println!("{}", from_utf8_basic(&res.msg_data.body)?);
  http2.send_go_away(Http2ErrorCode::NoError).await;
  Ok(())
}

Server Example

//! Low-level HTTP/2 server that servers a single response.

extern crate tokio;
extern crate wtx;
extern crate wtx_examples;

use tokio::net::TcpListener;
use wtx::{
  collections::Vector,
  http::{HttpRecvParams, Response, StatusCode},
  http2::{Http2, Http2Buffer, Http2ErrorCode, Http2RecvStatus},
  misc::Uri,
  rng::{ChaCha20, CryptoSeedableRng},
  stream::Stream,
  tls::{TlsAcceptor, TlsConfig, TlsModeVerified},
};
use wtx_examples::{PUBLIC_KEY, SECRET_KEY, host_from_args};

#[tokio::main]
async fn main() -> wtx::Result<()> {
  let uri = Uri::new(host_from_args());
  let listener = TcpListener::bind(uri.hostname_with_implied_port()).await?;
  let (stream, _) = listener.accept().await?;
  let mut rng = ChaCha20::from_getrandom()?;
  let hb = Http2Buffer::new(&mut rng);
  let tls_stream = TlsAcceptor::new(
    TlsConfig::from_keys_pem(TlsModeVerified::default(), PUBLIC_KEY, SECRET_KEY)?,
    rng,
    stream,
  )
  .accept()
  .await?
  .tls_stream;
  let hrp = HttpRecvParams::with_optioned_params();
  let (frame_reader, http2) = Http2::accept(hb, hrp, tls_stream.into_split()?).await?;
  let _jh = tokio::spawn(frame_reader);
  let Some((mut stream, _)) = http2.stream(|_, _| {}).await? else {
    println!("Connection closed!");
    return Ok(());
  };
  let (hrs, msg) = stream.recv_req().await?;
  if let Http2RecvStatus::ClosedConnection | Http2RecvStatus::ClosedStream(_) = hrs {
    println!("Connection or stream closed!");
    return Ok(());
  }
  println!("An arbitrary request has been received: {msg:#?}");
  let _ = stream
    .send_res(&mut Vector::new(), Response::new(b"By tea, for tea\n", StatusCode::ImATeapot))
    .await?;
  http2.send_go_away(Http2ErrorCode::NoError).await;
  Ok(())
}

HTTP/2 Client Pool

High-level pool of HTTP clients where multiple connections that can be referenced in concurrent scenarios.

Reuses valid connections and recycles dropped communications to minimize contention and latency. Instances are created on-demand and maintained for subsequent requests to the same host.

Also useful because HTTP/2 and HTTP/3 expect long-lived sessions by default unlike HTTP/1.

To use this functionality, it is necessary to activate the http2-client-pool feature.

Example

//! Fetches and prints the response body of a provided URI.

extern crate tokio;
extern crate wtx;

use wtx::{
  collections::Vector,
  executor::TokioExecutor,
  http::{HttpClient, ReqBuilder, http2_client_pool::Http2ClientPoolBuilder},
  misc::{Uri, from_utf8_basic},
  rng::{ChaCha20, CryptoSeedableRng as _},
  tls::{TlsConfig, TlsModeVerified},
};

#[tokio::main]
async fn main() -> wtx::Result<()> {
  let uri = Uri::new("https://github.com/c410-f3r/wtx");
  let res = Http2ClientPoolBuilder::new(
    TokioExecutor::default(),
    1,
    ChaCha20::from_getrandom()?,
    TlsConfig::from_ccadb(TlsModeVerified::default())?,
  )?
  .build()
  .send_req_recv_res(&mut Vector::new(), ReqBuilder::get(uri.to_ref()).into_request())
  .await?;
  println!("{}", from_utf8_basic(&res.msg_data.body)?);
  Ok(())
}

HTTP/2 Server Framework

A small and fast to compile framework that can interact with many built-in features.

• Databases
• JSON
• Middlewares
• Streaming
• URI router
• WebSocket

To use this functionality, it is necessary to activate the http2-server-framework feature.

Endpoints

Under the hood every endpoint transforms a Request (Body, Method, Headers, Uri) into a Response (Body, Headers, StatusCode) and users can perform a finer control over this process.

Input

You will get what you declare as input. Everything else is previously sanitized.

#![allow(unused)]
fn main() {
async fn health() {}
}

The above example accepts nothing () as input parameter, which automatically implies a sanitization of the received request.

#![allow(unused)]
fn main() {
extern crate wtx;

use wtx::http::http2_server_framework::State;

async fn print_request(state: State<'_, ()>) {
  assert_eq!(state.req.msg_data.body.len(), 0);
}
}

On the other hand, the above example gives you access to the full request. This also implies that you should be responsable for data management.

Output

Determines how responses are constructed. Similar to input handling, the output defines what clients receive.

For instance, this endpoint returns a simple Hello as the response body alongside an implicit 200 OK status. No headers are sent.

#![allow(unused)]
fn main() {
async fn hello() -> &'static str {
  "Hello"
}
}

There are many other types of outputs that perform different operations. Please see the documentation for a full listening.

Buffers

A key consideration is that the buffers used for receiving requests are the same ones utilized for constructing responses, which means, among other things:

1. If the response body is equal to or smaller than the request body in size, a memory allocation can be avoided, potentially improving runtime performance.
2. If you use State with VerbatimParams or DynParams::Verbatim, then you should probably be careful to avoid leaking request information into responses.

#![allow(unused)]
fn main() {
extern crate wtx;

use wtx::http::http2_server_framework::{State, VerbatimParams};

// The response will contain the same headers and data received from the request. Basically an echo.
async fn echo(_: State<'_, ()>) -> wtx::Result<VerbatimParams> {
  Ok(VerbatimParams::default())
}
}

Example

//! An HTTP/2 server framework showcasing nested routes, middlewares, manual streams, dynamic routes,
//! PostgreSQL connections and JSON deserialization/serialization.

extern crate serde;
extern crate serde_json;
extern crate tokio;
extern crate wtx;
extern crate wtx_examples;

use core::{fmt::Write, ops::ControlFlow};
use tokio::net::{TcpStream, tcp::OwnedWriteHalf};
use wtx::{
  database::{DbClient, Record},
  http::{
    ManualStream, Method, MsgBufferString, MsgDataMut, Request, Response, StatusCode,
    http2_server_framework::{
      Http2ServerFramework, HttpRouter, JsonReply, Middleware, Path, State, StateClean,
      VerbatimParams, get, json,
    },
  },
  http2::{Http2ErrorCode, ServerStream},
  misc::SecretContext,
  pool::{PostgresRM, SimplePool},
  rng::{ChaCha20, CryptoSeedableRng},
  tls::{TlsConfig, TlsModeVerified},
};
use wtx_examples::{PUBLIC_KEY, ROOT_CA, SECRET_KEY, host_from_args};

type LocalPool = SimplePool<PostgresRM<wtx::Error, TcpStream, TlsModeVerified>>;

fn main() -> wtx::Result<()> {
  let mut uri = *b"postgres://USER:PASSWORD@localhost/DB_NAME";
  let mut server = Http2ServerFramework::tokio(TlsConfig::from_keys_pem(
    TlsModeVerified::default(),
    PUBLIC_KEY.try_into()?,
    SECRET_KEY.try_into()?,
  )?)?;
  let pool = LocalPool::new(
    4,
    PostgresRM::new(
      ChaCha20::from_crypto_rng(server.rng_mut())?,
      SecretContext::new(server.rng_mut())?,
      TlsConfig::from_trust_anchors_pem(TlsModeVerified::default(), [ROOT_CA])?,
      &mut uri,
    )?,
  );
  let router = HttpRouter::paths(wtx::paths!(
    ("/db/{id}", get(db)),
    ("/json", json(Method::Post, deserialization_and_serialization)),
    (
      "/say",
      HttpRouter::new(
        wtx::paths!(("/hello", get(hello)), ("/world", get(world))),
        CustomMiddleware,
      )?,
    ),
    ("/stream", get(stream)),
  ))?;
  server
    .set_data(pool)
    .set_error_cb(|err| eprintln!("Error: {err}"))
    .run_in_threads(&host_from_args(), router)
}

async fn db(state: StateClean<'_, LocalPool>, Path(id): Path<u32>) -> wtx::Result<VerbatimParams> {
  let mut lock = state.data.get_with_unit().await?;
  let record = lock.execute_stmt_single("SELECT name FROM persons WHERE id = $1", (id,)).await?;
  let name = record.decode::<_, &str>(0)?;
  state.req.msg_data.body.write_fmt(format_args!("Person of id `{id}` has name `{name}`"))?;
  Ok(VerbatimParams(StatusCode::Ok))
}

async fn deserialization_and_serialization(state: State<'_, LocalPool>) -> wtx::Result<JsonReply> {
  let deserialize_example: DeserializeExample = serde_json::from_slice(&state.req.msg_data.body)?;
  let serialize_example = SerializeExample {
    _baz: [u32::from(deserialize_example._bar / 2), u32::from(deserialize_example._bar % 2)],
  };
  state.req.msg_data.clear();
  serde_json::to_writer(&mut state.req.msg_data.body, &serialize_example)?;
  Ok(JsonReply::default())
}

async fn hello() -> &'static str {
  "hello"
}

async fn stream(
  mut manual_stream: ManualStream<LocalPool, ServerStream<OwnedWriteHalf, TlsModeVerified>>,
) -> wtx::Result<()> {
  manual_stream.stream.common().send_go_away(Http2ErrorCode::NoError).await;
  Ok(())
}

async fn world() -> &'static str {
  "world"
}

struct CustomMiddleware;

impl Middleware<LocalPool, wtx::Error> for CustomMiddleware {
  type Aux = ();

  fn aux(&self) -> Self::Aux {}

  async fn req(
    &self,
    _: &mut LocalPool,
    _: &mut Self::Aux,
    _: &mut Request<MsgBufferString>,
  ) -> wtx::Result<ControlFlow<StatusCode, ()>> {
    println!("Inspecting request");
    Ok(ControlFlow::Continue(()))
  }

  async fn res(
    &self,
    _: &mut LocalPool,
    _: &mut Self::Aux,
    _: Response<&mut MsgBufferString>,
  ) -> wtx::Result<ControlFlow<StatusCode, ()>> {
    println!("Inspecting response");
    Ok(ControlFlow::Continue(()))
  }
}

#[derive(serde::Deserialize)]
struct DeserializeExample {
  _foo: u16,
  _bar: u16,
}

#[derive(serde::Serialize)]
struct SerializeExample {
  _baz: [u32; 2],
}

Internal Development

Intended for the development of WTX although some tips might be useful for your projects.

Size constraints

A large enum aggressively used in several places can cause a negative runtime impact. In fact, this is so common that the community created several lints to prevent such a scenario.

• large_enum_variant
• result_large_err
• variant_size_differences

Some real-world use-cases and associated benchmarks.

• https://ziglang.org/download/0.8.0/release-notes.html#Reworked-Memory-Layout
• https://github.com/rust-lang/rust/pull/100441
• https://github.com/rust-lang/rust/pull/95715

That is why WTX has an enforced Error enum size of 16 bytes and that is also the reason why WTX has so many bare error variants.

Performance

Many things that generally improve performance are used in the project, to name a few:

1. Manual Vectorization: When an algorithm is known for processing large amounts of data, several experiments are performed to analyze the best way to split loops in order to allow the compiler to take advantage of SIMD instructions.
2. Memory Allocation: Whenever possible, all structures related to heap allocations are only created at the instantiation level.
3. Fewer Dependencies: No third-party is injected by default. In other words, additional dependencies are up to the user through the selection of Cargo features, which decreases the compilation time of full builds. For example, you can see the mere 7 dependencies required by the PostgreSQL client using cargo tree -e normal --features crypto-ring,postgres.
4. Vectored and Buffered IO: Instead of writing a single chunk of data and waiting for it to be sent, multiple chunks are gathered and transmitted in a single operation whenever possible.

Connection Management

All protocols are expected to end an connection with a signal that allows a graceful stop. For example, when WebSocket sends a Close frame.

Sequential code

• Local termination: Sends a termination signal that halts the writing of further data. Remote actor is expected to also send a termination signal within a timeout. Internal state jumps from Open to WriteClosed.
• Remote termination: Receives a termination signal that halts the reading of further data. Immediately sends a terminal signal and closes the connection. Internal state jumps from Open to Closed.

Concurrent code

• Local termination: Sends a termination signal that halts the writing of further data. Remote actor is expected to also send a termination signal within a timeout. Internal state jumps from Open to WriteClosed.
• Remote termination: Receives a termination signal that halts the reading of further data. Local system will certainly send a termination signal in a posterior step. Internal state jumps from Open to ReadClosed.

Internal state is expected to jump from ReadClosed or WriteClosed to Closed once the termination cycle is fulfilled.

Protocols

Methods associated to external reads return optional elements to reflect that a connection can be closed anytime locally or by the peer. However, there are 2 exceptions to this rule.

• Handshakes: Graceful stops are not expected in initial handshakes.
• PostgreSQL: Clients expect that a connection will never be closed by the database.

Profiling

Uses the h2load benchmarking tool (https://nghttp2.org/documentation/h2load-howto.html) and the h2load internal binary (https://github.com/c410-f3r/wtx/blob/main/wtx-internal/src/bin/h2load.rs) for illustration purposes.

Compilation time / Size

cargo-bloat: Finds out what takes most of the space in executables.

cargo bloat --bin h2load --features h2load | head -20

cargo-llvm-lines: Measures the number and size of instantiations of each generic function in a program.

CARGO_PROFILE_RELEASE_LTO=fat cargo llvm-lines --bin h2load --features h2load --package wtx-internal --release | head -20

Performance

Prepare the executables in different terminals.

h2load -c100 --log-file=/tmp/h2load.txt -m10 -n10000 --no-tls-proto=h2c http://localhost:9000

cargo build --bin h2load --features h2load --profile profiling --target x86_64-unknown-linux-gnu

samply: Command line CPU profiler.

samply record ./target/x86_64-unknown-linux-gnu/profiling/h2load

callgrind: Gives global, per-function, and per-source-line instruction counts and simulated cache and branch prediction data.

valgrind --tool=callgrind --dump-instr=yes --collect-jumps=yes --simulate-cache=yes ./target/x86_64-unknown-linux-gnu/profiling/h2load

RFCs

List of implemented RFCs.

Compiler flags

Some non-standard options that will influence the final binary. Only use them if you know what you are doing.

Size

• -C force-frame-pointers=no
• -C force-unwind-tables=no

More size-related parameters can be found at https://github.com/johnthagen/min-sized-rust.

Runtime

• -C llvm-args=–inline-threshold=9999
• -C llvm-args=-enable-dfa-jump-thread
• -C llvm-args=-vectorize-loops
• -C llvm-args=-vectorize-slp
• -C target-cpu=x86-64-v3

Security

• -C control-flow-guard=yes
• -C relocation-model=pie
• -C relro-level=full
• -Z stack-protector=strong

Secrets

The Secret struct is a container of sensitive data that needs to be sustained in memory for an extended period. More specifically, it holds locked and encrypted heap-allocated bytes that are decrypted on demand to protect against inspection techniques.

Please keep in mind that this is not a silver bullet, but rather an additional layer of protection. For example, when the peek closure is executing, the plaintext secret will exist transiently in CPU registers and caches, which is unavoidable.

Example

//! Long lived secret

extern crate wtx;

use crate::wtx::rng::CryptoSeedableRng;
use std::{env, sync::OnceLock};
use wtx::{
  collections::Vector,
  misc::{Secret, SecretContext},
  rng::ChaCha20,
};

static SECRET: OnceLock<Secret> = OnceLock::new();

fn main() -> wtx::Result<()> {
  let data = env::args()
    .nth(1)
    .ok_or_else(|| wtx::Error::GenericStatic("No data".try_into().unwrap_or_default()))?;
  let mut rng = ChaCha20::from_getrandom()?;
  let secret_context = SecretContext::new(&mut rng)?;
  let secret = Secret::new(data.into_bytes().as_mut(), &mut rng, secret_context)?;
  let _rslt = SECRET.set(secret);
  std::thread::spawn(|| {
    let mut buffer = Vector::new();
    SECRET.wait().peek(&mut buffer, |_data| {
      // Sign documents, pass API keys, etc...
    })?;
    wtx::Result::Ok(())
  })
  .join()??;
  Ok(())
}

TLS

TLS 1.3 is the only supported version.

Implementation of RFC-8446.

Transport Layer Security (TLS) is a cryptographic protocol that provides secure communication over a computer network by encrypting data to ensure confidentiality, integrity, and authentication. It is widely used in applications such as web browsers ensuring that contents transferred between parties can not be intercepted or altered by unauthorized actors.

To use this functionality, it is necessary to activate the tls feature.

Plain-text

It is possible to convert a TLS stream into an unencrypted stream through the use of the TlsModePlainText structure. In other words, TlsModePlainText makes the TLS stream act like a normal plain-text stream ignoring all associated certificates, handshakes and encryptions.

This feature is useful for local tests and also for applications running behind a service mesh that automatically handles mTLS connections. However, TlsModePlainText can be ***DANGEROUS*** in a misconfiguration or if you don’t know what are you doing, as such, be careful!

Robustness

Our TLS stack is brand new so if you encountered any error, feel free to open an issue.

On its own, the TLS 1.3 RFC is huge, complex and prone to errors. Not to mention other associated features like ECH or DTLS.

To allow a reliable implementation, WTX is trying to integrate the boringssl testsuite as well as the testssl tool in a slow but steady pace. Hopefully everything will be much safer in the next few months.

Concurrency

The RFC requires all parties (Client or Server) to send back carefully managed records, such as alerts, if an error occurs.

WTX automatically enforces these rules in sequential code but how is the reader part going to access the writer part in concurrent scenarios? In fact, there are numerous ways to approach this and the choice is yours to make.

Examples about possible concurrent utilizations are available in the wtx-examples directory.

Example

//! TLS client that reads and writes records.

extern crate tokio;
extern crate wtx;

use tokio::net::TcpStream;
use wtx::{
  misc::process_utf8_stream,
  rng::{ChaCha20, CryptoSeedableRng as _},
  stream::{StreamReader, StreamWriter},
  tls::{TlsConfig, TlsConnector, TlsModeVerified},
};

#[tokio::main]
async fn main() -> wtx::Result<()> {
  let stream = TcpStream::connect("github.com:443").await?;
  let tls_config = TlsConfig::from_ccadb(TlsModeVerified::default())?;
  let tls_connector = TlsConnector::new(tls_config, ChaCha20::from_getrandom()?, stream);
  let mut tls_stream = tls_connector.connect().await?.tls_stream;
  let request = b"GET /c410-f3r/wtx HTTP/1.1\r\nHost: github.com\r\nConnection: close\r\n\r\n";
  tls_stream.write_all(request).await?;
  let mut partial_char = None;
  loop {
    let mut buffer = [0; 128];
    let Some(read) = tls_stream.read(buffer.as_mut_slice().into()).await? else {
      return Ok(());
    };
    let slice = buffer.get(..read.get()).unwrap_or_default();
    let (lhs, rhs) = process_utf8_stream(&mut partial_char, slice)?;
    println!("{lhs}{rhs}");
  }
}

UI Tools

wtx-ui is a standalone crate intended to allow interactions with the wtx project through an user interface. At the current time only CLI interfaces are available.

• Embeds SQL migrations for schema-manager. Activation feature is called embed-migrations.
• Runs SQL migrations managed by schema-manager. Activation feature is called schema-manager or schema-manager-dev.
• Performs very basic WebSocket Client/Server operations. Activation feature is called web-socket.
• Makes requests to arbitrary URIs mimicking the interface of cURL. Activation feature is called http-client.

WebSocket

Implementation of RFC6455 and RFC7692. WebSocket is a communication protocol that enables full-duplex communication between a client (typically a web browser) and a server over a single TCP connection. Unlike traditional HTTP, which is request-response based, WebSocket allows real-time data exchange without the need for polling.

In-house benchmarks are available at https://c410-f3r.github.io/wtx-bench. If you are aware of other benchmark tools, please open a discussion in the GitHub project.

To use this functionality, it is necessary to activate the web-socket feature.

Autobahn Reports

1. fuzzingclient
2. fuzzingserver

Compression

The “permessage-deflate” extension is the only supported compression format and is backed by the zlib-rs project that performs as well as zlib-ng.

At the current time WTX is the only crate that allows lock-free reader and writer parts with compression support.

To get the most performance possible, try compiling your program with RUSTFLAGS='-C target-cpu=native' to allow zlib-rs to use more efficient SIMD instructions.

No masking

Although not officially endorsed, the no-masking parameter described at https://datatracker.ietf.org/doc/html/draft-damjanovic-websockets-nomasking-02 is supported to increase performance. If such a thing is not desirable, please make sure to check the handshake parameters to avoid accidental scenarios.

To make everything work as intended both parties, client and server, need to implement this feature. For example, web browsers won’t stop masking frames.

Ping and Close frames

A received Ping frame automatically triggers an internal Pong response. Similarly, when a Close frame is received an automatic Close frame response is also sent.

//! WebSocket client that reads and writes frames in the same task.

extern crate tokio;
extern crate wtx;
extern crate wtx_examples;

use tokio::net::TcpStream;
use wtx::{
  collections::Vector,
  misc::Uri,
  rng::{ChaCha20, CryptoSeedableRng},
  tls::{TlsConfig, TlsConnector, TlsModeVerified},
  web_socket::{OpCode, WebSocketConnector, WebSocketPayloadOrigin},
};
use wtx_examples::{ROOT_CA, uri_from_args};

#[tokio::main]
async fn main() -> wtx::Result<()> {
  let uri = Uri::new(uri_from_args());
  let stream = TcpStream::connect(uri.hostname_with_implied_port()).await?;
  let mut ws = WebSocketConnector::default()
    .connect(
      TlsConnector::new(
        TlsConfig::from_trust_anchors_pem(TlsModeVerified::default(), [ROOT_CA])?,
        ChaCha20::from_getrandom()?,
        stream,
      ),
      &uri.to_ref(),
    )
    .await?;
  let mut buffer = Vector::new();
  loop {
    let frame = ws.read_frame(&mut buffer, WebSocketPayloadOrigin::Adaptive).await?;
    match (frame.op_code(), frame.text_payload()) {
      // `read_frame` internally already sent a Close response
      (OpCode::Close, _) => {
        break;
      }
      // `read_frame` internally already sent a Pong response
      (OpCode::Ping, _) => {}
      // For any other type, `read_frame` doesn't automatically send frames
      (_, text) => {
        if let Some(elem) = text {
          println!("Received text frame: {elem}")
        }
      }
    }
  }
  Ok(())
}

The same automatic behavior does not happen with concurrent instances because there are multiple ways to synchronize resources. In other words, you are responsible for managing replies.

//! WebSocket client that reads and writes frames in different tasks.
//!
//! Special frames aren't automatically handled by the system in concurrent scenarios because there are
//! multiple ways to synchronize resources. In this example, special frames are managed using a
//! mutex but you can utilize any other method.
//!
//! `wtx-client-concurrent` is an example that uses a channel.

extern crate tokio;
extern crate wtx;
extern crate wtx_examples;

use tokio::net::TcpStream;
use wtx::{
  collections::Vector,
  misc::Uri,
  rng::{ChaCha20, CryptoSeedableRng as _},
  sync::{Arc, AsyncMutex},
  tls::{TlsConfig, TlsConnector, TlsModeVerified},
  web_socket::{Frame, OpCode, WebSocketConnector, WebSocketPayloadOrigin},
};
use wtx_examples::{ROOT_CA, uri_from_args};

#[tokio::main]
async fn main() -> wtx::Result<()> {
  let uri = Uri::new(uri_from_args());
  let stream = TcpStream::connect(uri.hostname_with_implied_port()).await?;
  let ws = WebSocketConnector::default()
    .connect(
      TlsConnector::new(
        TlsConfig::from_trust_anchors_pem(TlsModeVerified::default(), [ROOT_CA])?,
        ChaCha20::from_getrandom()?,
        stream,
      ),
      &uri.to_ref(),
    )
    .await?;
  let (stream_bridge, mut stream_reader, stream_writer) = ws.into_split()?;
  let stream_writer_bridge = Arc::new(AsyncMutex::new(stream_writer));
  let stream_writer_writer = stream_writer_bridge.clone();

  let bridge_fut = async {
    loop {
      let data = stream_bridge.listen().await;
      if stream_writer_bridge.lock().await.manage_bridge_data(data).await? {
        break;
      }
    }
    wtx::Result::Ok(())
  };

  let reader_fut = async {
    let mut buffer = Vector::new();
    loop {
      let frame = stream_reader.read_frame(&mut buffer, WebSocketPayloadOrigin::Adaptive).await?;
      match (frame.op_code(), frame.text_payload()) {
        // A special version of this frame has already been sent to the bridge
        (OpCode::Close, _) => break,
        // A `Pong` frame with the same content has already been sent to the bridge
        (OpCode::Ping, _) => {}
        (_, text) => {
          if let Some(elem) = text {
            println!("Received text frame: {elem}")
          }
        }
      }
    }
    wtx::Result::Ok(())
  };

  let writer_fut = async {
    stream_writer_writer
      .lock()
      .await
      .write_frame(&mut Frame::new_fin(OpCode::Close, *b"Bye")?)
      .await?;
    wtx::Result::Ok(())
  };

  let (bridge_rslt, reader_rslt, writer_rslt) = tokio::join!(bridge_fut, reader_fut, writer_fut);
  bridge_rslt?;
  reader_rslt?;
  writer_rslt?;
  Ok(())
}

Alternative replying methods can be found at web-socket in the wtx-examples crate.

Server Example

//! Serves requests using low-level WebSockets resources alongside self-made certificates.

extern crate tokio;
extern crate wtx;
extern crate wtx_examples;

use tokio::net::TcpListener;
use wtx::{
  collections::Vector,
  rng::{ChaCha20, CryptoSeedableRng},
  tls::{TlsAcceptor, TlsConfig, TlsModeVerified},
  web_socket::{OpCode, WebSocketAcceptor, WebSocketPayloadOrigin},
};
use wtx_examples::{PUBLIC_KEY, SECRET_KEY, host_from_args};

#[tokio::main]
async fn main() -> wtx::Result<()> {
  let listener = TcpListener::bind(&host_from_args()).await?;
  let mut rng = ChaCha20::from_getrandom()?;
  loop {
    let conn_rng = ChaCha20::from_crypto_rng(&mut rng)?;
    let (stream, _) = listener.accept().await?;
    let _jh = tokio::spawn(async move {
      let fut = async {
        let mut buffer = Vector::new();
        let mut ws = WebSocketAcceptor::default()
          .accept(TlsAcceptor::new(
            TlsConfig::from_keys_pem(TlsModeVerified::default(), PUBLIC_KEY, SECRET_KEY)?,
            conn_rng,
            stream,
          ))
          .await?;
        let (mut common, mut reader, mut writer) = ws.split_mut();
        loop {
          let origin = WebSocketPayloadOrigin::Adaptive;
          let mut frame = reader.read_frame(&mut buffer, &mut common, origin).await?;
          match frame.op_code() {
            OpCode::Binary | OpCode::Text => writer.write_frame(&mut common, &mut frame).await?,
            OpCode::Close => break,
            _ => {}
          }
        }
        wtx::Result::Ok(())
      };
      if let Err(err) = fut.await {
        eprintln!("Error: {err}");
      }
    });
  }
}

WebSocket over HTTP/2

At the current time only servers support the handshake procedure defined in RFC-8441.

While HTTP/2 inherently supports full-duplex communication, web browsers typically don’t expose this functionality directly to developers and that is why WebSocket tunneling over HTTP/2 is important.

1. Servers can efficiently handle multiple concurrent streams within a single TCP connection
2. Client applications can continue using existing WebSocket APIs without modification

For this particular scenario the no-masking parameter defined in https://datatracker.ietf.org/doc/html/draft-damjanovic-websockets-nomasking-02 is also supported.

To use this functionality, it is necessary to activate the http2 and web-socket features.

Example

//! Low-level HTTP/2 server that servers a single response.

extern crate tokio;
extern crate wtx;
extern crate wtx_examples;

use tokio::net::TcpListener;
use wtx::{
  collections::Vector,
  http::{HttpRecvParams, Response, StatusCode},
  http2::{Http2, Http2Buffer, Http2ErrorCode, Http2RecvStatus},
  misc::Uri,
  rng::{ChaCha20, CryptoSeedableRng},
  stream::Stream,
  tls::{TlsAcceptor, TlsConfig, TlsModeVerified},
};
use wtx_examples::{PUBLIC_KEY, SECRET_KEY, host_from_args};

#[tokio::main]
async fn main() -> wtx::Result<()> {
  let uri = Uri::new(host_from_args());
  let listener = TcpListener::bind(uri.hostname_with_implied_port()).await?;
  let (stream, _) = listener.accept().await?;
  let mut rng = ChaCha20::from_getrandom()?;
  let hb = Http2Buffer::new(&mut rng);
  let tls_stream = TlsAcceptor::new(
    TlsConfig::from_keys_pem(TlsModeVerified::default(), PUBLIC_KEY, SECRET_KEY)?,
    rng,
    stream,
  )
  .accept()
  .await?
  .tls_stream;
  let hrp = HttpRecvParams::with_optioned_params();
  let (frame_reader, http2) = Http2::accept(hb, hrp, tls_stream.into_split()?).await?;
  let _jh = tokio::spawn(frame_reader);
  let Some((mut stream, _)) = http2.stream(|_, _| {}).await? else {
    println!("Connection closed!");
    return Ok(());
  };
  let (hrs, msg) = stream.recv_req().await?;
  if let Http2RecvStatus::ClosedConnection | Http2RecvStatus::ClosedStream(_) = hrs {
    println!("Connection or stream closed!");
    return Ok(());
  }
  println!("An arbitrary request has been received: {msg:#?}");
  let _ = stream
    .send_res(&mut Vector::new(), Response::new(b"By tea, for tea\n", StatusCode::ImATeapot))
    .await?;
  http2.send_go_away(Http2ErrorCode::NoError).await;
  Ok(())
}

X.509

The most utilized format to define public key certificates. X.509 is used in TLS connections, e-mail communication, internal corporate structuring and many other cases.

Implementation of https://datatracker.ietf.org/doc/html/rfc5280. Passes a subset of the x509-limbo testsuite (https://github.com/C2SP/x509-limbo).

To use this functionality, it is necessary to activate the x509 feature.

Unsupported

X.509 is huge and was initially issued in 1988, consequently, it contains some elements that are rarely used nowadays. To facilitate development and create convergence, the following items won’t be supported as indicated by the CABF (https://cabforum.org).

• Authority Key Identifier: authorityCertIssuer and authorityCertSerialNumber.

• GeneralSubtree: minimum and maximum.

Example

//! X.509 chain validation

extern crate wtx;

use core::slice;
use wtx::{
  asn1::parse_der_from_pem_range,
  calendar::DateTime,
  codec::{Decode as _, DecodeWrapper},
  collections::Vector,
  misc::Pem,
  x509::{Certificate, CvEndEntity, CvIntermediate, CvPolicy, CvTrustAnchor},
};

const END_ENTITY: &[u8] = b"-----BEGIN CERTIFICATE-----
MIIFxDCCBWmgAwIBAgIQaD3YAMfWDUsaC+cNmW6poDAKBggqhkjOPQQDAjBRMQsw
CQYDVQQGEwJVUzETMBEGA1UEChMKQXBwbGUgSW5jLjEtMCsGA1UEAxMkQXBwbGUg
UHVibGljIEVWIFNlcnZlciBFQ0MgQ0EgMSAtIEcxMB4XDTI2MDIyNjE4MDcxNloX
DTI2MDUyNzE5MDk0OVowgcMxHTAbBgNVBA8MFFByaXZhdGUgT3JnYW5pemF0aW9u
MRMwEQYLKwYBBAGCNzwCAQMTAlVTMRswGQYLKwYBBAGCNzwCAQIMCkNhbGlmb3Ju
aWExETAPBgNVBAUTCEMwODA2NTkyMQswCQYDVQQGEwJVUzETMBEGA1UECAwKQ2Fs
aWZvcm5pYTESMBAGA1UEBwwJQ3VwZXJ0aW5vMRMwEQYDVQQKDApBcHBsZSBJbmMu
MRIwEAYDVQQDDAlhcHBsZS5jb20wWTATBgcqhkjOPQIBBggqhkjOPQMBBwNCAATF
8Ac3KZPGRdrd5PtJta681gjvUlVA7qR+iWw3/JB+PI48vOoYzTw86Z/W6hrlmcoQ
S6pRj03OEmmcWXkTB1e4o4IDrjCCA6owDAYDVR0TAQH/BAIwADAfBgNVHSMEGDAW
gBTghUh9E6bTEBmfXMtreCSS+K4brjB6BggrBgEFBQcBAQRuMGwwMgYIKwYBBQUH
MAKGJmh0dHA6Ly9jZXJ0cy5hcHBsZS5jb20vYXBldnNlY2MxZzEuZGVyMDYGCCsG
AQUFBzABhipodHRwOi8vb2NzcC5hcHBsZS5jb20vb2NzcDAzLWFwZXZzZWNjMWcx
MDEwFAYDVR0RBA0wC4IJYXBwbGUuY29tMGAGA1UdIARZMFcwSAYFZ4EMAQEwPzA9
BggrBgEFBQcCARYxaHR0cHM6Ly93d3cuYXBwbGUuY29tL2NlcnRpZmljYXRlYXV0
aG9yaXR5L3B1YmxpYzALBglghkgBhv1sAgEwEwYDVR0lBAwwCgYIKwYBBQUHAwEw
NQYDVR0fBC4wLDAqoCigJoYkaHR0cDovL2NybC5hcHBsZS5jb20vYXBldnNlY2Mx
ZzEuY3JsMB0GA1UdDgQWBBQjqdcO+ifokErbZnTjOrNR1SgNiTAOBgNVHQ8BAf8E
BAMCB4AwDwYJKoZIhvdjZAZWBAIFADCCAfcGCisGAQQB1nkCBAIEggHnBIIB4wHh
AHYAlpdkv1VYl633Q4doNwhCd+nwOtX2pPM2bkakPw/KqcYAAAGcmythYgAABAMA
RzBFAiEA1PhX+2cAc4EW0geuD07k34wvWCOzfzcDbkN4IACLxCECIFm2mf+KrYxz
POfCTAy19oRfnya5HVqBytwL+MPUgFlxAHcAZBHEbKQS7KeJHKICLgC8q08oB9Qe
NSer6v7VA8l9zfAAAAGcmythHwAABAMASDBGAiEAyZ8Fg+8QrV0CaXhIMZexYTdM
D2KOaUJf7SwF3DvDWR0CIQCyPDXDHMgaAC8oBPHSRBIzbE9M2LkqpNcpOq/yThzZ
KQB2ABaDLavwqSUPD/A6pUX/yL/II9CHS/YEKSf45x8zE/X6AAABnJsrYUMAAAQD
AEcwRQIgJbU/WVxUUOdhNCy/UpEjbqYFI1NXQrIyuvqIJNTWMGUCIQDzCVTxeChI
em7lS5ISRhcbbwbznmKNIaFh2f3ITWDGRgB2AMs49xWJfIShRF9bwd37yW7ymlnN
RwppBYWwyxTDFFjnAAABnJsrYToAAAQDAEcwRQIgP3cimH+o3cyCPT9yPyqS39H+
ec0utyDDYBSr45J4BPMCIQD4KNwoJWADwMAWuSfjAL9sxTH2hHdAA0gjnKrRPYT7
GjAKBggqhkjOPQQDAgNJADBGAiEAtsvO+N4V5m9WAsC12+qaJS5WFhTxoVmy/b34
v4Y96TACIQCCLj3iOuotRmOrHyF6XXT6/XD5Lj4Yjd5pbOFmNLQ8vg==
-----END CERTIFICATE-----";

const INTERMEDIATE: &[u8] = b"-----BEGIN CERTIFICATE-----
MIIDsjCCAzigAwIBAgIQDKuq0c7E6XzCZliB0CE49zAKBggqhkjOPQQDAzBhMQsw
CQYDVQQGEwJVUzEVMBMGA1UEChMMRGlnaUNlcnQgSW5jMRkwFwYDVQQLExB3d3cu
ZGlnaWNlcnQuY29tMSAwHgYDVQQDExdEaWdpQ2VydCBHbG9iYWwgUm9vdCBHMzAe
Fw0yMDA0MjkxMjM0NTJaFw0zMDA0MTAyMzU5NTlaMFExCzAJBgNVBAYTAlVTMRMw
EQYDVQQKEwpBcHBsZSBJbmMuMS0wKwYDVQQDEyRBcHBsZSBQdWJsaWMgRVYgU2Vy
dmVyIEVDQyBDQSAxIC0gRzEwWTATBgcqhkjOPQIBBggqhkjOPQMBBwNCAAQp+OFa
uYdEBJj/FpCG+eDhQmVfhv0DGPzGz40TW8BeWxipYTOa4FLieAYoU+3t2tg9FZKt
A4BDTO43YprLZm6zo4IB4DCCAdwwHQYDVR0OBBYEFOCFSH0TptMQGZ9cy2t4JJL4
rhuuMB8GA1UdIwQYMBaAFLPbSKT5ocXYrjZBzBFjaWIpvEvGMA4GA1UdDwEB/wQE
AwIBhjAdBgNVHSUEFjAUBggrBgEFBQcDAQYIKwYBBQUHAwIwEgYDVR0TAQH/BAgw
BgEB/wIBADA0BggrBgEFBQcBAQQoMCYwJAYIKwYBBQUHMAGGGGh0dHA6Ly9vY3Nw
LmRpZ2ljZXJ0LmNvbTBCBgNVHR8EOzA5MDegNaAzhjFodHRwOi8vY3JsMy5kaWdp
Y2VydC5jb20vRGlnaUNlcnRHbG9iYWxSb290RzMuY3JsMIHcBgNVHSAEgdQwgdEw
gcUGCWCGSAGG/WwCATCBtzAoBggrBgEFBQcCARYcaHR0cHM6Ly93d3cuZGlnaWNl
cnQuY29tL0NQUzCBigYIKwYBBQUHAgIwfgx8QW55IHVzZSBvZiB0aGlzIENlcnRp
ZmljYXRlIGNvbnN0aXR1dGVzIGFjY2VwdGFuY2Ugb2YgdGhlIFJlbHlpbmcgUGFy
dHkgQWdyZWVtZW50IGxvY2F0ZWQgYXQgaHR0cHM6Ly93d3cuZGlnaWNlcnQuY29t
L3JwYS11YTAHBgVngQwBATAKBggqhkjOPQQDAwNoADBlAjEAyHLAT/4iBuxi4/NH
hZde4PZO8CnG2/A3oGO0Nsjpoe2SV94Hr+JpYHrBzT8hyeKSAjBnRXyRac9sM8KN
Fdg3+7LWIiW9sUjtJC6kGmRyGm6vV4oAhEDd9jdk4q+7b5zlid4=
-----END CERTIFICATE-----";

const TRUST_ANCHOR: &[u8] = b"-----BEGIN CERTIFICATE-----
MIICPzCCAcWgAwIBAgIQBVVWvPJepDU1w6QP1atFcjAKBggqhkjOPQQDAzBhMQsw
CQYDVQQGEwJVUzEVMBMGA1UEChMMRGlnaUNlcnQgSW5jMRkwFwYDVQQLExB3d3cu
ZGlnaWNlcnQuY29tMSAwHgYDVQQDExdEaWdpQ2VydCBHbG9iYWwgUm9vdCBHMzAe
Fw0xMzA4MDExMjAwMDBaFw0zODAxMTUxMjAwMDBaMGExCzAJBgNVBAYTAlVTMRUw
EwYDVQQKEwxEaWdpQ2VydCBJbmMxGTAXBgNVBAsTEHd3dy5kaWdpY2VydC5jb20x
IDAeBgNVBAMTF0RpZ2lDZXJ0IEdsb2JhbCBSb290IEczMHYwEAYHKoZIzj0CAQYF
K4EEACIDYgAE3afZu4q4C/sLfyHS8L6+c/MzXRq8NOrexpu80JX28MzQC7phW1FG
fp4tn+6OYwwX7Adw9c+ELkCDnOg/QW07rdOkFFk2eJ0DQ+4QE2xy3q6Ip6FrtUPO
Z9wj/wMco+I+o0IwQDAPBgNVHRMBAf8EBTADAQH/MA4GA1UdDwEB/wQEAwIBhjAd
BgNVHQ4EFgQUs9tIpPmhxdiuNkHMEWNpYim8S8YwCgYIKoZIzj0EAwMDaAAwZQIx
AK288mw/EkrRLTnDCgmXc/SINoyIJ7vmiI1Qhadj+Z4y3maTD/HMsQmP3Wyr+mt/
oAIwOWZbwmSNuJ5Q3KjVSaLtx9zRSX8XAbjIho9OjIgrqJqpisXRAL34VOKa5Vt8
sycX
-----END CERTIFICATE-----";

fn main() -> wtx::Result<()> {
  let mut buffer = Vector::new();

  let end_entity_range = Pem::decode(&mut DecodeWrapper::new(END_ENTITY, &mut buffer))?;
  let intermediate_range = Pem::decode(&mut DecodeWrapper::new(INTERMEDIATE, &mut buffer))?;
  let trust_anchor_range = Pem::decode(&mut DecodeWrapper::new(TRUST_ANCHOR, &mut buffer))?;

  let end_entity_params = parse_der_from_pem_range(&buffer, &end_entity_range)?;
  let intermediate_params = parse_der_from_pem_range(&buffer, &intermediate_range)?;
  let trust_anchor: Certificate<&[u8]> = parse_der_from_pem_range(&buffer, &trust_anchor_range)?.0;

  validate_chain(
    &CvEndEntity::from_certificate(end_entity_params.0, end_entity_params.1)?,
    &CvIntermediate::from_certificate(intermediate_params.0, intermediate_params.1)?,
    &CvTrustAnchor::from_certificate_ref(&trust_anchor)?,
  )
}

fn validate_chain<'any>(
  end_entity: &CvEndEntity<&'any [u8]>,
  intermediate: &CvIntermediate<&'any [u8]>,
  trust_anchor: &CvTrustAnchor<&'any [u8]>,
) -> wtx::Result<()> {
  let cvp = CvPolicy::new(DateTime::from_timestamp_secs(1779000000)?);
  let verified_path = end_entity.validate_chain(
    slice::from_ref(intermediate),
    &cvp,
    slice::from_ref(trust_anchor),
  )?;
  assert_eq!(verified_path.end_entity(), end_entity);
  assert_eq!(verified_path.intermediates(), &[intermediate]);
  assert_eq!(verified_path.trust_anchor(), trust_anchor);
  Ok(())
}