rust-bindgen Code Review

发表于 2016-10-02 | 分类于 World of Rust |

Abstract

In this blog, I will review the code of rust-bindgen, a binding tool for binding c library to rust. I’ll start with the binding concepts, then I will explain you the idea behind the project, and the details of how it works.

binding

Concepts of Binding

a binding from a programming language to a library or operating system service is an application programming interface (API) providing glue code to use that library or service in a particular programming language [binding-wiki]

Two related Problems [binding-acm]:

the mapping of specific language features/capabilities to the interface.
how the language bindings is documented as a standard.

Mapping Interfaces

[binding-acm]

direct mapping, lexically substitute source identifiers for the equivalent target identifiers.
- simple to implement.
- not using strong type definition and exception mechanism
abstract mapping, starts with the API’s interface semantics, and then produces a representation of the abstract interface in terms appropriate to the language at hand.
- Make full use of entire semantics of target language
- Requires human analysis

Documentation for Binding

thin document, cite the previous document.
thick document, describe the full semantics by appropriate language binding document.

Review of `rust-bindgen`

Idea behind the project

Rust-bindgen is a native binding generator for the rust language, originally created by Jyun-Yan You(crabtw), now maintained by Yamakaky. This project was originally ported from clay’s bindgen

The general idea behind the project is AST translation :

Use Clang to retrieve AST for source files.
Transverse the AST and mark the node with entity information (Convert node to global entity).
Generate new definitions for each entity, make new AST items.
Rewrite the code to Rust.

Project structure of rust-bindgen

The project directive is shown as below.

+-- _Cargo.toml
+-- _src
|   +-- clang.rs
|   +-- gen.rs
|   +-- lib.rs
|   +-- main.rs
|   +-- parser.rs
|   +-- types.rs

clangs.rs is a wrapper of clang related objects, Cursor, Type (CXType), TranslationUnit, etc.
types.rs defines representations of top level entity (Global), C types(Type).
parse.rs transverse the AST, and mark the AST with Global information, the result is a vector of globals. (type definition, composite type, enum, global variable, function prototype and else)
gen.rs, classify the globals vector into fs(GFunc), vs and gs, extract definitions and function, and make new AST items (Translation). The result is definitions of the new items (defs), and attributes of the module.
lib.rs, the higher wrapper for Bindings and builder, it use the syntex_syntax::print::pprust in the write function to write the defs to rust source file.

Intermediate Running Result of rust-bindgen

source code
```
int (*foo) (int x, int y)
```
globals, after parsing
```
"foo"
```

defs, after generating

Item { ident: #0, attrs: [], id: 4294967295, node: ForeignMod(ForeignMod { abi: C, items: [ForeignItem { ident: foo#0, attrs: [], node: Static(type(::std::option::Option<extern "C" fn(x: ::std::os::raw::c_int,
                                    y: ::std::os::raw::c_int)

rust source, after writing.

extern "C" {
    pub static mut foo:
               ::std::option::Option<extern "C" fn(x: ::std::os::raw::c_int,
                                                   y: ::std::os::raw::c_int)
                                         -> ::std::os::raw::c_int>;
}

Translation Rules for Function

The processing of a function definition:

call extract_functions (gen.rs) to extract (abi, ast)
- match the ty field of GFunc, if it is TFuncPtr go to 2.
  - call the cfunc_to_rs function to convert
    
    make name by rust_id
    
    extract attrs by mk_link_name_attr
    
    call mk_foreign_item to return an ast item.
  - return the ast item
call mk_extern to make ast::Item

Translation Rules for Types and Consts

Decide whether it is a type or const

The processing of a consts:
- vs =⇒ cvar_to_rs
  - make name by rust_id
  - extract attrs by mk_link_name_attr
  - call mk_foreign_item to return an ast item.
The processing of types
- call extract_definition for gs,
  - GType
  - GCompDecl
  - GComp
  - GEnumDecl
  - GEnum
  - GVar

Analysis of Corrode

https://github.com/jameysharp/corrode

This program reads a C source file and prints an equivalent module in Rust syntax.

Written in Haskell
Follow the idea of translating AST.
1. Use language-c to extract the command-line arguments we care about. We’ll pass the rest to the preprocessor.
2. The user may have specified the -o <outputfile> option. Not only do we ignore that, but we need to suppress it so the preprocessor doesn’t write its output where a binary was expected to be written. We also force-undefine preprocessor symbols that would indicate support for language features we can’t actually handle, and remove optimization flags that make GCC define preprocessor symbols.
3. Run the preprocessor—except that if the input appears to have already been preprocessed, then we should just read it as-is.
4. Get language-c to parse the preprocessed source to a CTranslUnit.
5. Generate a list of Rust items from this C translation unit.
6. Pretty-print all the items as a String.
7. Write the final string to a file with the same name as the input, except with any extension replaced by ".rs".

Reference

[binding-wiki] https://en.wikipedia.org/wiki/Language_binding
[binding-acm] David Emery. Standards, APIs, Interfaces and Bindings. http://oldwww.acm.org/tsc/apis.html
[ast-instrument] adamrehn. AST Instrumentation (examples by language). http://adamrehn.com/articles/ast-instrumentation-examples-by-language/
[basic-tranform-clang] Eli Bendersky, Basic source-to-source transformation with Clang, http://eli.thegreenplace.net/2012/06/08/basic-source-to-source-transformation-with-clang/
[transform-clang] Zellescher Weg, Performing Source-to-Source Transformations with Clang, http://llvm.org/devmtg/2013-04/krzikalla-slides.pdf
[globalanalysis] Diomidis Spinellis, Global Analysis and Transformations in Preprocessed Languages, http://www.spinellis.gr/pubs/jrnl/2003-TSE-Refactor/html/Spi03r.html

Homebrew Related Errors and Solutions

发表于 2016-09-30 | 分类于 tools |

Abstract

This blog collects the errors and solutions I met with brew.

. dyld: Library not loaded:

1
2
3

* dyld: Library not loaded: /usr/local/opt/readline/lib/libreadline.6.dylib
  Referenced from: /usr/local/bin/bash
    Reason: image not found

This occurs because of /usr/local/bin/bash requires libreadline.6, but there is not such a dylib in the directory. This happens may be because the dependency was installed after bash, but it doesn’t update the bash dependency, so a solution for this is :

1 2	brew rm bash brew install bash

Configure Hexo to Support Asciidoctor

发表于 2016-09-30 | 分类于 tools |

Startup

Please follow the guide on hexo official documents

Some good guides in Chinese are(很好的中文教程):

Crossin’s GitHub+Hexo建立你的第一个博客

I use deck.js to render slides.

Supporting Asciidoctor

install asciidoctor.js

cd /path/to/hexo-site-dir/
npm install asciidoctor.js --save

install hexo-renderer-asciidoc

cd /path/to/hexo-site-dir/
npm install hexo-renderer-asciidoc --save

RVM Ruby Configuration on Mac

发表于 2016-09-30 | 分类于 tools |

Abstract

This blog talks about ruby configuration.

Basics

rvm, ruby version manager, is used to intall Ruby and manage your Rails versions.
ruby, is the interpreter for the Ruby programming language.
rails, is a web application development framework.
gems, is the soft libraries.
1
2
gem install **
gem uninstall **
gemset is to store different gems for different purposes. You can use rvm gemset list to show all the gemset. By default, there are two gemsets, one is default, one is global. You can use rvm gemset use global to swith to global gemset. When you install gem in global gemset, it will come use to all other gemset.

Install

follow the guide in InstallRubyonRailsmac

Usages

remove rvm
1
2
rvm implode
rm -rf ~/.rvm

Errors

. bundle install returns Could not locate Gemfile or .bundle/ directory
You just need to change directories to your app, THEN run bundle install :) Stackoverflow

A Vim IDE for RUST language

发表于 2016-09-12 | 分类于 World of Rust |

Abstract

In order to make an IDE for rust, I followed the Ultimate vimrc, and installed Tagbar, YouCompleteMe to support the RUST language.

Details

Install the Ultimate vimrc, you can follow the readme.

1 2	git clone https://github.com/amix/vimrc.git ~/.vim_runtime sh ~/.vim_runtime/install_awesome_vimrc.sh

Install Your Plugins to the directory sources_non_forked.

1 2	cd ~/vim_runtime git clone git://github.com/tpope/vim-rails.git sources_non_forked/vim-rails

Install tagbar

1	git clone https://github.com/majutsushi/tagbar.git sources_non_forked/tagbar

Config tagbar for Rust, Put this into ~/.ctags

--langdef=Rust
--langmap=Rust:.rs
--regex-Rust=/^[ \t]*(#\[[^\]]\][ \t]*)*(pub[ \t]+)?(extern[ \t]+)?("[^"]+"[ \t]+)?(unsafe[ \t]+)?fn[ \t]+([a-zA-Z0-9_]+)/\6/f,functions,function definitions/
--regex-Rust=/^[ \t]*(pub[ \t]+)?type[ \t]+([a-zA-Z0-9_]+)/\2/T,types,type definitions/
--regex-Rust=/^[ \t]*(pub[ \t]+)?enum[ \t]+([a-zA-Z0-9_]+)/\2/g,enum,enumeration names/
--regex-Rust=/^[ \t]*(pub[ \t]+)?struct[ \t]+([a-zA-Z0-9_]+)/\2/s,structure names/
--regex-Rust=/^[ \t]*(pub[ \t]+)?mod[ \t]+([a-zA-Z0-9_]+)/\2/m,modules,module names/
--regex-Rust=/^[ \t]*(pub[ \t]+)?(static|const)[ \t]+(mut[ \t]+)?([a-zA-Z0-9_]+)/\4/c,consts,static constants/
--regex-Rust=/^[ \t]*(pub[ \t]+)?(unsafe[ \t]+)?trait[ \t]+([a-zA-Z0-9_]+)/\3/t,traits,traits/
--regex-Rust=/^[ \t]*(pub[ \t]+)?(unsafe[ \t]+)?impl([ \t\n]*<[^>]*>)?[ \t]+(([a-zA-Z0-9_:]+)[ \t]*(<[^>]*>)?[ \t]+(for)[ \t]+)?([a-zA-Z0-9_]+)/\5 \7 \8/i,impls,trait implementations/
--regex-Rust=/^[ \t]*macro_rules![ \t]+([a-zA-Z0-9_]+)/\1/d,macros,macro definitions/

and add configuration to the my_configs.vim as step 4.

Install YouCompleteMe for rust follow the README:

cd ~/.vim_runtime/sources_non_forked
git clone https://github.com/Valloric/YouCompleteMe.git sources_non_forked/YouCompleteMe
cd YouCompleteMe
git submodule update --init --recursive
./install.py --racer-completer
mkdir -p ~/Developer
mkdir -p ~/Developer
cd ~/Developer
git clone --depth 1 --branch master https://github.com/rust-lang/rust rust-master

Add your configurations to the file ~/vim_runtime/my_configs.vim.

"YouCompleteMe for rust
let g:ycm_rust_src_path="/home/huang/Developer/rust-master/src/"
"enable mouse support
set mouse=a
"Arrage the NERDTreePanel to the right
let NERDTreeWinPos = "right"
"Arrage the tagbar to the left
let g:tagbar_left = 1
let g:tagbar_width = 30
let g:tagbar_autofocus = 1
let g:tagbar_sort = 0
let g:tagbar_compact = 1
"enable tagbar for rust
let g:tagbar_type_rust = {
    \ 'ctagstype' : 'rust',
    \ 'kinds' : [
        \'T:types,type definitions',
        \'f:functions,function definitions',
        \'g:enum,enumeration names',
        \'s:structure names',
        \'m:modules,module names',
        \'c:consts,static constants',
        \'t:traits,traits',
        \'i:impls,trait implementations',
    \]
    \}
"shortcuts
nmap <F5> :TagbarToggle<cr>
nmap <F6> :NERDTreeToggle<cr>

Other Resources

[1]: Awesome Rust Collections, https://github.com/kud1ing/awesome-rust#ides

[2]: Vim Cheet Sheet, http://vim.rtorr.com/

[3]: VimAwesome, http://vimawesome.com/

Abstract

binding

Concepts of Binding

Mapping Interfaces

Documentation for Binding

Review of rust-bindgen

Idea behind the project

Project structure of rust-bindgen

Intermediate Running Result of rust-bindgen

Translation Rules for Function

Translation Rules for Types and Consts

Analysis of Corrode

Reference

Abstract

Startup

Supporting Asciidoctor

Abstract

Basics

Install

Usages

Errors

Abstract

Details

Other Resources

Review of `rust-bindgen`