rust-bindgen Code Review

发表于   |   分类于   |  

Abstract

In this blog, I will review the code of rust-bindgen, a binding tool for binding c library to rust. I’ll start with the binding concepts, then I will explain you the idea behind the project, and the details of how it works.

binding

Concepts of Binding

a binding from a programming language to a library or operating system service is an application programming interface (API) providing glue code to use that library or service in a particular programming language [binding-wiki]

Two related Problems [binding-acm]:

  1. the mapping of specific language features/capabilities to the interface.

  2. how the language bindings is documented as a standard.

Mapping Interfaces

[binding-acm]

  • direct mapping, lexically substitute source identifiers for the equivalent target identifiers.

    • simple to implement.

    • not using strong type definition and exception mechanism

  • abstract mapping, starts with the API’s interface semantics, and then produces a representation of the abstract interface in terms appropriate to the language at hand.

    • Make full use of entire semantics of target language

    • Requires human analysis

Documentation for Binding

  • thin document, cite the previous document.

  • thick document, describe the full semantics by appropriate language binding document.

Review of rust-bindgen

Idea behind the project

Rust-bindgen is a native binding generator for the rust language, originally created by Jyun-Yan You(crabtw), now maintained by Yamakaky. This project was originally ported from clay’s bindgen

The general idea behind the project is AST translation :

  • Use Clang to retrieve AST for source files.

  • Transverse the AST and mark the node with entity information (Convert node to global entity).

  • Generate new definitions for each entity, make new AST items.

  • Rewrite the code to Rust.

Project structure of rust-bindgen

The project directive is shown as below.

+-- _Cargo.toml
+-- _src
|   +-- clang.rs
|   +-- gen.rs
|   +-- lib.rs
|   +-- main.rs
|   +-- parser.rs
|   +-- types.rs

  • clangs.rs is a wrapper of clang related objects, Cursor, Type (CXType), TranslationUnit, etc.

  • types.rs defines representations of top level entity (Global), C types(Type).

  • parse.rs transverse the AST, and mark the AST with Global information, the result is a vector of globals. (type definition, composite type, enum, global variable, function prototype and else)

  • gen.rs, classify the globals vector into fs(GFunc), vs and gs, extract definitions and function, and make new AST items (Translation). The result is definitions of the new items (defs), and attributes of the module.

  • lib.rs, the higher wrapper for Bindings and builder, it use the syntex_syntax::print::pprust in the write function to write the defs to rust source file.

Intermediate Running Result of rust-bindgen

  1. source code

    int (*foo) (int x, int y)

  2. globals, after parsing

    "foo"

  3. defs, after generating

    Item { ident: #0, attrs: [], id: 4294967295, node: ForeignMod(ForeignMod { abi: C, items: [ForeignItem { ident: foo#0, attrs: [], node: Static(type(::std::option::Option<extern "C" fn(x: ::std::os::raw::c_int,
                                    y: ::std::os::raw::c_int)

  4. rust source, after writing.

    extern "C" {
    pub static mut foo:
               ::std::option::Option<extern "C" fn(x: ::std::os::raw::c_int,
                                                   y: ::std::os::raw::c_int)
                                         -> ::std::os::raw::c_int>;
}

Translation Rules for Function

The processing of a function definition:

  • call extract_functions (gen.rs) to extract (abi, ast)

    • match the ty field of GFunc, if it is TFuncPtr go to 2.

      • call the cfunc_to_rs function to convert

        • make name by rust_id

        • extract attrs by mk_link_name_attr

        • call mk_foreign_item to return an ast item.

      • return the ast item

  • call mk_extern to make ast::Item

Translation Rules for Types and Consts

Decide whether it is a type or const

  1. The processing of a consts:

    • vs =⇒ cvar_to_rs

      • make name by rust_id

      • extract attrs by mk_link_name_attr

      • call mk_foreign_item to return an ast item.

  2. The processing of types

    • call extract_definition for gs,

      • GType

      • GCompDecl

      • GComp

      • GEnumDecl

      • GEnum

      • GVar

Analysis of Corrode

https://github.com/jameysharp/corrode

This program reads a C source file and prints an equivalent module in Rust syntax.

  • Written in Haskell

  • Follow the idea of translating AST.

    1. Use language-c to extract the command-line arguments we care about. We’ll pass the rest to the preprocessor.

    2. The user may have specified the -o <outputfile> option. Not only do we ignore that, but we need to suppress it so the preprocessor doesn’t write its output where a binary was expected to be written. We also force-undefine preprocessor symbols that would indicate support for language features we can’t actually handle, and remove optimization flags that make GCC define preprocessor symbols.

    3. Run the preprocessor—except that if the input appears to have already been preprocessed, then we should just read it as-is.

    4. Get language-c to parse the preprocessed source to a CTranslUnit.

    5. Generate a list of Rust items from this C translation unit.

    6. Pretty-print all the items as a String.

    7. Write the final string to a file with the same name as the input, except with any extension replaced by ".rs".

Reference