Danblog

Medical Resident, Software Developer, Basketball Fan

Writing a Simple Web Service in Rust

I'm a Rust beginner, but it's quickly becoming my favorite language to use. While writing small projects in Rust is usually a little less ergonomic and more time consuming (with me behind the wheel, at least), it challenges the way I think about program design. My fights with the compiler are becoming less frequent; when they do occur, I come away having learned something.

I've been working on the zigbee2mqtt Hass.io add-on, an extension for the Home Assistant home automation platform. The add-on relies on the zigbee2mqtt library. zigbee2mqtt is fairly new, iterating rapidly, and not yet using versioned releases. Hass.io add-ons are distributed as Docker images, and the zigbee2mqtt add-on simply checks out the latest master branch of the underlying library when building a Docker image. This approach posed a problem: when new commits were pushed to zigbee2mqtt, users of the add-on were not able to update to the latest version until the add-on image was rebuilt (which happens automatically within Travis CI only when commits are pushed to the add-on repository). I needed a way to trigger add-on builds on Travis whenever the library was modified on Github. Why not Rust?

In this post, I'll walk through creating a simple web service in Rust using actix-web that accepts incoming Github webhook posts and triggers a Travis CI build via the Travis API V3.

Capturing a Github Webhook

Once webhooks are configured, the Github API v3 sends a PushEvent to a specified URL with a JSON payload with plenty of information about the commit (described in the docs). For the purposes of this example, we only really care about the ref field, from which we can get branch information:

/// An incoming PushEvent from Github Webhook.
#[derive(Deserialize)]
struct PushEvent {
    #[serde(rename = "ref")]
    reference: String,
}

serde helps us deserialize the payload into the PushEvent struct, and actix_web's extractors make it extremely easy to access JSON data in a handler function. A handler function can accept some type Json<T> as an argument, and the request body will automatically be deserialized into type T, so long as it implements serde's Deserialize:

use actix_web::Json;

fn index(push: Json<PushEvent>) -> ...

Within the handler, the request body is automatically deserialized into the push argument, which has type PushEvent.

Checking Headers with Middleware

Github webhooks can be accompanied by a secret for authentication. If provided, the secret will be used to create an HMAC-SHA1 of the request body, and sent with the request via the X-Hub-Signature header in the format "sha1=<HMAC>". We can implement middleware with actix_web to check this header for each request:

use actix_web::HttpRequest;
use actix_web::middleware::{Middleware, Started};
use actix_web::error::{ErrorUnauthorized, ParseError};
use actix_web::Result;

struct VerifySignature;

impl<S> Middleware<S> for VerifySignature {
    fn start(&self, req: &mut HttpRequest<S>) -> Result<Started> {
        use std::io::Read;

        let r = req.clone();
        let s = r.headers()
            .get("X-Hub-Signature")
            .ok_or(ErrorUnauthorized(ParseError::Header))?
            .to_str()
            .map_err(ErrorUnauthorized)?;
        // strip "sha1=" from the header
        let (_, sig) = s.split_at(5);

        let secret = env::var("GITHUB_SECRET").unwrap();
        let mut body = String::new();
        req.read_to_string(&mut body)
            .map_err(ErrorInternalServerError)?;

        if is_valid_signature(&sig, &body, &secret) {
            Ok(Started::Done)
        } else {
            Err(ErrorUnauthorized(ParseError::Header))
        }
    }
}

is_valid_signature is a function similar to that defined by @aergonaut (see this blog post for more). Using the crytpo crate, we compare the signature sent in the X-Hub-Signature to what we calculate from the request body and our known secret. The only major differences are how the hexstring is constructed (does not require unsafe) and how the sha1= prefix is handled.

Thanks to /u/vbrandl on Reddit for teaching me how this works.

Using the Travis API

The Travis V3 API exposes the /repo/{slug|id}/requests endpoint, which allows for triggering new builds. Taking from the documentation, here's the basic request that we need to implement:

body='{
"request": {
"message": "Override the commit message: this is an api request",
"branch":"master"
}}'

curl -s -X POST \
   -H "Content-Type: application/json" \
   -H "Accept: application/json" \
   -H "Travis-API-Version: 3" \
   -H "Authorization: token xxxxxx" \
   -d "$body" \
   https://api.travis-ci.com/repo/danielwelch%2Fhassio-zigbee2mqtt/requests

hyper provides a nice macro that allows us to define custom headers while maintaining a degree of type safety.

header! { (TravisAPIVersion, "Travis-API-Version") => [u16] }

Now we can add our headers and JSON body with reqwest::RequestBuilder.

#[derive(Serialize)]
struct TravisRequest {
    message: String,
    branch: String,
}

fn travis_request(url: &str) -> Result<reqwest::Response> {
    let client = reqwest::Client::new();
    let res = client
        .post(url)
        .header(reqwest::header::ContentType::json())
        .header(TravisAPIVersion(3))
        .header(reqwest::header::Authorization(auth_str()))
        .json(&TravisRequest {
            message: "API Request triggered by zigbee2mqtt update".to_string(),
            branch: "master".to_string(),
        })
        .send()
        .map_err(ErrorInternalServerError)?;
    Ok(res)

fn auth_str() -> String {
    format!("token {}", std::env::var("TRAVIS_TOKEN").unwrap()).to_owned()
}

Designing a Responder

In actix-web, a handler just needs to implement Handler, which is already implemented for any function that takes an HttpRequest and returns some Responder. Json<T> implements FromRequest, which converts an HttpRequest to Json<T> behind the scenes. Taking the first part of our function definition above, we can now finish it out, knowing what we need to return.

fn index(push: Json<PushEvent>) -> impl Responder {} // very fancy and new impl in return position

All that's left to do is implement some Responder according to the docs. This will be a JSON-serialized message that will be sent in response to a successful request.

use actix_web::{Responder, HttpRequest, HttpResponse, Error};

#[derive(Serialize)]
struct ServerMessage(String) 

impl Responder for ServerMessage {
    type Item = HttpResponse;
    type Error = Error;

    fn respond_to<S>(self, _req: &HttpRequest<S>) -> Result<HttpResponse, Error> {
        let body = serde_json::to_string(&self)?;
        Ok(HttpResponse::Ok()
            .content_type("application/json")
            .body(body))
    }
}

Putting it all together as an endpoint:

use std::env;
use actix_web::{Json, Responder, HttpRequest, HttpResponse, Error};
use actix_web::error::ErrorInternalServerError;

fn index(push: Json<PushEvent>) -> impl Responder {
    let travis_url = env::var("TRAVIS_URL").unwrap();
    if push.reference.ends_with("master") {
        match travis_request("https://api.travis-ci.org/repo/19145006/requests") {
            Ok(_) => ServerMessage(format!(
                "PushEvent on branch master found, request sent to {}",
                travis_url).to_owned()),
            Err(e) => ErrorInternalServerError(e),
        }
    } else {
        ServerMessage("PushEvent is not for master branch".to_owned())
    }
}

This won't work. The compiler complains of mismatched arms in the match statement. The actix-web docs suggest using the Either type to return two different types from a Responder.

use actix_web::Either;

type ServerResponse = Either<ServerMessage, Error>;

fn index(push: Json<PushEvent>) -> impl Responder {
    let travis_url = env::var("TRAVIS_URL").unwrap();
    if push.reference.ends_with("master") {
        match travis_request("https://api.travis-ci.org/repo/19145006/requests") {
            Ok(_) => Either::A(
                ServerMessage(format!(
                    "PushEvent on branch master found, request sent to {}",
                    travis_url)
                .to_owned())
            ),
            Err(e) => Either::B(ErrorInternalServerError(e)),
        }
    } else {
        Either::A(ServerMessage("PushEvent is not for master branch".to_owned()))
    }
}

This works, but I don't like it. It's ugly and seems too complex for such a simple case. There has to be a better way than this to return either a serialized message or an HTTP error response from an actix-web resource. And, after reading more of the documentation, I discovered there are many better ways. I was interested in one that made use of my Responder implementation for ServerMessage, and kept most of the response handler logic tied to that struct, because it made me feel cool. In fact, my Responder implementation's respond_to method is already prepared to return a Result of either an HttpResponse or an Error. Why don't we handle the error within the SeverMessage struct and its reponse implementation?

#[derive(Serialize)]
struct ServerMessage {
    message: String,

    // we don't need to serialize the error,
    // it will be transmitted instead of the 
    // serialized ServerMessage HTTPResponse (below)
    #[serde(skip_serializing)]
    e: Option<Error>,
}


impl Responder for ServerMessage {
    type Item = HttpResponse;
    type Error = Error;

    fn respond_to<S>(self, _req: &HttpRequest<S>) -> Result<HttpResponse, Error> {
        if self.e.is_some() {
            return Err(self.e.unwrap());
        } else {
            let body = serde_json::to_string(&self)?;
            Ok(HttpResponse::Ok()
                .content_type("application/json")
                .body(body))
        }
    }
}

So an error can be captured within the struct and handled during a response. The error is defined at some other point in the request-response cycle--it doesn't matter where or what the error is, as long as it's an actix-web::Error. Adding some convenience methods to make things a little cleaner...

impl ServerMessage {
    fn success<T: ToString>(s: T) -> ServerMessage {
        ServerMessage {
            message: s.to_string(),
            e: None,
        }
    }

    fn error(e: Error) -> ServerMessage {
        ServerMessage {
            message: "".to_owned(),
            e: Some(e),
        }
    }
}

And our final endpoint looks much better:

fn index(push: Json<PushEvent>) -> impl Responder {
    let travis_url = env::var("TRAVIS_URL").unwrap();
    if push.reference.ends_with("master") {
        match travis_request("https://api.travis-ci.org/repo/19145006/requests") {
            Ok(_) => ServerMessage::success(format!(
                "PushEvent on branch master found, request sent to {}",
                travis_url
            )),
            Err(e) => ServerMessage::error(e),
        }
    } else {
        ServerMessage::success("PushEvent is not for master branch")
    }
}

All that's left to do is start up the server in main.rs.

/// utility function from the heroku buildpack example project
fn get_server_port() -> u16 {
    env::var("PORT")
        .ok()
        .and_then(|p| p.parse().ok())
        .unwrap_or(8080)
}

fn main() {
    use std::net::{SocketAddr, ToSocketAddrs};
    let sys = actix::System::new("updater");
    let addr = SocketAddr::from(([0, 0, 0, 0], get_server_port()));

    server::new(|| {
        App::new()
            .middleware(HeaderCheck)
            .resource("/", |r| r.method(http::Method::POST).with(index))
    }).bind(addr)
        .unwrap()
        .start();

    let _ = sys.run();
}

Conclusion

Well, it turns out that it was easier to just grant write access to the author of zigbee2mqtt and have them trigger a build via an after_success script in that repository. Nevertheless, this was a fun exercise in the basics of a new Rust web framework, and some new concepts it introduces. See the full repository here.

Feedback Welcome