David Dong's Blog

Rust startup - trait

David Dong — Thu, 15 Feb 2024 11:22:40 +0800

Is Rust an object-oriented programming language? I have searched for the answer to this question on different AI assistants.

Below are the answers. Let’s see.

CHATGPT 3.5

While Rust supports some object-oriented programming (OOP) concepts, it is not strictly classified as an object-oriented programming language. Rust is more commonly categorized as a multi-paradigm programming language, combining features from OOP, functional programming, and other paradigms.

And

Copilot

Rust is not strictly an object-oriented programming language.

However, it shares some features with OOP, making it a fascinating blend of paradigms.

And

Gemini

The question of whether Rust is an object-oriented programming language (OOP) has a nuanced answer. While it shares some features with OOP languages, it ultimately has a different paradigm. So, is Rust OOP?

From a strict definition: Technically no, Rust’s core design philosophy differs from traditional OOP.

In practice: It borrows certain OOP concepts and tools, offering a familiar feel for OOP programmers.

More accurately: Rust uses a multi-paradigm approach, incorporating elements of OOP, functional programming, and others.

So the next question is how the Rust achieves polymorphism. The answer is using traits.

In Rust, we can define data structures using structs and enums, and achieve polymorphism using traits and trait objects, which share some similarities with classes and interfaces in OOP. However, Rust does not have classes or inheritance as found in traditional OOP languages. Instead, Rust emphasizes ownership, lifetimes, and abstraction through traits and generics.

Below is an example of how to use traits.

trait allows developers to define shared behavior that can be implemented by different types. They serve as a way to express interfaces or contracts that types must adhere to.

trait Animal {
    fn speak(&self);
}

struct Dog {
    name: String,
    age: u32,
}

impl Animal for Dog {
    fn speak(&self) {
        println!("Woof!");
    }
}

struct Cat {
    name: String,
    age: u32,
}

impl Animal for Cat {
    fn speak(&self) {
        println!("Meow!");
    }
}

fn do_speak(animal : &dyn Animal){
    animal.speak();
}

fn main() {

    let dog = Dog {name: String::from("kaka"), age: 10};
    let cat = Cat {name: String::from("haha"), age:5};

    println!("{} is {} years old", dog.name, dog.age);
    println!("{} is {} years old", cat.name, cat.age);

    dog.speak();
    cat.speak();

    do_speak(&dog);
    do_speak(&cat);
}

Output

kaka is 10 years old
haha is 5 years old
Woof!
Meow!
Woof!
Meow!

Rust startup - Ownership

David Dong — Wed, 14 Feb 2024 11:20:15 +0800

Ownership in Rust refers to a set of rules that govern how memory is managed. At any given time, each value in Rust has a variable that is known as its owner. There can be only one owner at a time. When the owner goes out of scope, the value is dropped and its memory is deallocated.

Ownership ensures that memory is used safely and efficiently, and it prevents common memory errors such as dangling pointers and memory leaks.

The Ownership Rules

The ownership rules in Rust are as follows:

Every value in Rust has an owner. The owner is the variable or expression that is last assigned to the value.
There can only be one owner of a value at a time. When a value is assigned to a new variable, the ownership of the value is transferred to the new variable.
When a value goes out of scope, the value is dropped. The value is deallocated and its memory is returned to the system.

The Benefits of Ownership

Ownership provides several benefits for Rust programs, including:

Memory safety: Ownership prevents common memory errors such as dangling pointers and memory leaks.
Efficiency: Ownership allows Rust to optimize memory usage by deallocating memory as soon as it is no longer needed.
Expressiveness: Ownership can be used to express the lifetime of data in Rust programs.

Ownership and Borrowing

In some cases, it is necessary to allow multiple parts of a program to access the same data. This can be done using borrowing. Borrowing is a way to temporarily transfer ownership of a value to another variable or expression. References (&) are used to borrow values, and they have a limited lifetime during which they are valid.

Mutable Borrowing

Mutable references (&mut) allow you to borrow a value for both reading and writing. Only one mutable reference is allowed at a time in a given scope to prevent data races.

Clone

To create a new value with the same data without transferring ownership, you can explicitly clone the value. Cloning allocates new memory for the data, allowing multiple variables to own separate copies of the same data.

Ownership in Functions

Function parameters by default take ownership of the passed values. To borrow a value within a function without taking ownership, you can pass references as function arguments.

The Ownership Checker

The Rust compiler includes an ownership checker that helps to ensure that the ownership rules are followed. The ownership checker can detect and report errors such as dangling pointers and memory leaks.

Lifetimes

Lifetimes are annotations that specify the scope for which references are valid. They are used by the Rust compiler to enforce memory safety rules and prevent dangling references.

Ownership in Rust provides a powerful mechanism for memory management and ensures memory safety without the need for a garbage collector or manual memory management. It enables safe and efficient concurrency and system-level programming in Rust.

Example

struct Car {
    name: String,
    price: u32,
}
fn print_car(car: Car)
{
    println!("Name {}, price {}", car.name, car.price);
}
fn main() {

    let demo_car = Car {
        name: String::from("Ferrari"),
        price: 10000,
    };

    print_car(demo_car);
    //println!("Name {}, price {}", demo_car.name, demo_car.price);
}

Output

~ cargo run
   Compiling greeting v0.1.0 (/home/david/code/rust/first_proj/greeting)
    Finished dev [unoptimized + debuginfo] target(s) in 0.48s
     Running `target/debug/greeting`
Name Ferrari, price 10000

so the ownership of the demo_car has been transferred to function pirnt_car, try to use the demo_car after being transferred.

struct Car {
    name: String,
    price: u32,
}
fn print_car(car: Car)
{
    println!("Name {}, price {}", car.name, car.price);
}
fn main() {

    let demo_car = Car {
        name: String::from("Ferrari"),
        price: 10000,
    };

    print_car(demo_car);
    println!("Name {}, price {}", demo_car.name, demo_car.price);
}

as expected, a compile error occurred. The error encountered is due to the demo_car variable being moved when it’s passed to the print_car function. After the move, I am trying to use demo_car again in the println! macro, which is not allowed because ownership has already been transferred.

~ cargo run
   Compiling greeting v0.1.0 (/home/david/code/rust/first_proj/greeting)
error[E0382]: borrow of moved value: `demo_car`
  --> src/main.rs:17:50
   |
11 |     let demo_car = Car {
   |         -------- move occurs because `demo_car` has type `Car`, which does not implement the `Copy` trait
...
16 |     print_car(demo_car);
   |               -------- value moved here
17 |     println!("Name {}, price {}", demo_car.name, demo_car.price);
   |                                                  ^^^^^^^^^^^^^^ value borrowed here after move
   |
note: consider changing this parameter type in function `print_car` to borrow instead if owning the value isn't necessary
  --> src/main.rs:5:19
   |
5  | fn print_car(car: Car)
   |    ---------      ^^^ this parameter takes ownership of the value
   |    |
   |    in this function

To fix this, we can either clone the demo_car before passing it to print_car, or we can change the print_car function to accept a reference to Car rather than taking ownership of it. Here’s the modified code using the second approach:

#[derive(Clone)]
struct Car {
    name: String,
    price: u32,
}
fn print_car(car: Car)
{
    println!("Name {}, price {}", car.name, car.price);
}
fn main() {

    let demo_car = Car {
        name: String::from("Ferrari"),
        price: 10000,
    };

    print_car(demo_car.clone());
    println!("Name {}, price {}", demo_car.name, demo_car.price);
}

Output

~ cargo run
    Finished dev [unoptimized + debuginfo] target(s) in 0.05s
     Running `target/debug/greeting`
Name Ferrari, price 10000
Name Ferrari, price 10000

Above we clone demo_car before passing it to the print_car function.

In this version:

We’ve added #[derive(Clone)] to the Car struct definition to automatically derive the Clone trait for Car, allowing us to call the clone method on instances of Car.
When calling print_car, we pass a cloned copy of demo_car using demo_car.clone(). This creates a new copy of demo_car, and ownership of the copy is passed to the print_car function, allowing us to retain ownership of the original demo_car.
After calling print_car, we can still use demo_car as it has not been moved.

Below is another approach

struct Car {
    name: String,
    price: u32,
}
fn print_car(car: &Car)
{
    println!("Name {}, price {}", car.name, car.price);
}
fn main() {

    let demo_car = Car {
        name: String::from("Ferrari"),
        price: 10000,
    };

    print_car(&demo_car);
    println!("Name {}, price {}", demo_car.name, demo_car.price);
}

Output

~ cargo run
   Compiling greeting v0.1.0 (/home/david/code/rust/first_proj/greeting)
    Finished dev [unoptimized + debuginfo] target(s) in 0.35s
     Running `target/debug/greeting`
Name Ferrari, price 10000
Name Ferrari, price 10000

In this version:

The print_car function now accepts a reference to a Car struct using &Car as the parameter type.
Inside print_car, we use car.name and car.price to access the fields of the Car struct via the reference.
When calling print_car, we pass a reference to demo_car using &demo_car, which allows us to borrow demo_car without transferring ownership.
After calling print_car, we can still use demo_car as it has not been moved.

Rust startup - environment setup

David Dong — Sat, 10 Feb 2024 17:24:55 +0800

Rust is hot, I’ve heard a lot of people talking about the Rust language, and it’s powered me enough to learn more about it!

This is the first note to document the process of setting up the development environment, mainly choosing to develop under Linux.

Environments

OS - ubuntu 16.04
C Compiler - x86_64-unknown-linux-gnu
IDE - VS Code

Setup

The rust development environment consists of rustup (toolchain management tool), cargo (package manager and build system) and rustc (compiler).

$ export RUSTUP_DIST_SERVER=https://mirrors.ustc.edu.cn/rust-static
$ export RUSTUP_UPDATE_ROOT=https://mirrors.ustc.edu.cn/rust-static/rustup
$ sudo curl https://sh.rustup.rs -sSf | sh

choose option 1

& david @ david-VirtualBox in ~ 0 [15:51:08]
~ curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh                                                                 0 [15:51:08]
info: downloading installer

Welcome to Rust!

This will download and install the official compiler for the Rust
programming language, and its package manager, Cargo.

Rustup metadata and toolchains will be installed into the Rustup
home directory, located at:

  /home/david/.rustup

This can be modified with the RUSTUP_HOME environment variable.

The Cargo home directory is located at:

  /home/david/.cargo

This can be modified with the CARGO_HOME environment variable.

The cargo, rustc, rustup and other commands will be added to
Cargo's bin directory, located at:  /home/david/.cargo/bin 
This path will then be added to your PATH environment variable by modifying the 
profile files located at:  /home/david/.profile  /home/david/.bashrc 
You can uninstall at any time with rustup self uninstall and these changes will 
be reverted. 
Current installation options:   default 
host triple: x86_64-unknown-linux-gnu     default 
toolchain: stable (default)               
profile: default  
modify PATH variable: yes
1) Proceed with installation (default)
2) Customize installation
3) Cancel installation
>1
info: profile set to 'default'
info: default 
host triple is x86_64-unknown-linux-gnu
info: syncing channel updates for 'stable-x86_64-unknown-linux-gnu' 712.1 KiB
info: latest update on 2024-02-08, 
rust version 1.76.0 (07dca489a 2024-02-04)
info: downloading component 'cargo' 8.5 MiB / 8.5 MiB (100 %) 80.0 KiB/s in 1m 44s
info: downloading component 'clippy'  2.1 MiB /   2.1 MiB (100 %)  95.0 KiB/s in 23s
info: downloading component 'rust-docs' 14.7 MiB /  14.7 MiB (100 %) 117.4 KiB/s in 2m 
info: downloading component 'rust-std' 23.9 MiB /  23.9 MiB (100 %)  98.2 KiB/s in 4m 
info: downloading component 'rustc' 62.3 MiB /  62.3 MiB (100 %) 135.4 KiB/s in 12m 
info: downloading component 'rustfmt'  2.3 MiB /   2.3 MiB (100 %)  86.2 KiB/s in 30s 
info: installing component 'cargo'
info: installing component 'clippy'info: installing component 'rust-docs' 14.7 MiB 
info: installing component 'rust-std' 23.9 MiB /  23.9 MiB (100 %)  13.2 MiB/s in 1s
info: installing component 'rustc' 62.3 MiB /  62.3 MiB (100 %)  14.9 MiB/s in  4s 
info: installing component 'rustfmt'
info: default toolchain set to 'stable-x86_64-unknown-linux-gnu'  
stable-x86_64-unknown-linux-gnu installed 
- rustc 1.76.0 (07dca489a 2024-02-04)Rust is installed now. 
Great!To get started you may need to restart your current shell.
This would reload your PATH environment variable to include 
Cargo's bin directory ($HOME/.cargo/bin).

To configure your current shell, run:
source "$HOME/.cargo/env"

& david @ david-VirtualBox in ~ 0 [16:20:53]

The Rustup metadata and toolchain will be installed in the Rustup home directory located at /home/david/.rustup, which can be modified using the RUSTUP_HOME environment variable.
Cargo’s home directory is located at /home/david/.cargo which can be modified via the CARGO_HOME environment variable.
cargo, rustc, rustup and other commands will be added to cargo’s bin directory, which is located in /home/david/.cargo/bin
add this path to the PATH environment variable /home/david/.profile or /home/david/. bashrc
```
export PATH="$HOME/.cargo/bin:$PATH
```
I used fish shell, so need to add environment variable PATH in ~/.config/fish/config.fish
```
set -x PATH ~/.cargo/bin $PATH
```

Verify the installation

> rustc -V
rustc 1.76.0 (07dca489a 2024-02-04)
> cargo -V
cargo 1.76.0 (c84b36747 2024-01-18)
> rustup -V
rustup 1.26.0 (5af9b9484 2023-04-05)
info: This is the version for the rustup toolchain manager, not the rustc compiler.
info: The currently active `rustc` version is `rustc 1.76.0 (07dca489a 2024-02-04)`

The rust toolchain management tool rustup, the package manager cargo, and the compiler rustc are now installed!

unstallation command
```
rustup self uninstall
```

Vscode configuration for development of Rust

Install the following plug-ins

rust-analyzer
CodeLLDB
Code Runner

Vscode debugging Rust code

Create a project directory like first_prj
Go to first_prj

Run cargo new first_prj_name in terminal, it will create project first_prj_name directory and src/main. rs

Welcome to fish, the friendly interactive shell
Type help for instructions on how to use fish
  
& david @ ubuntu in ~/code/rust/first_proj 0 [19:06:14]
~ cargo new greeting
     Created binary (application) `greeting` package

cd . /first_prj_name

run cargo build to compile the code

& david @ ubuntu in ~/code/rust/first_proj/greeting (master …3) 0 [19:07:15]
~ cargo build
   Compiling greeting v0.1.0 (/home/david/code/rust/first_proj/greeting)
    Finished dev [unoptimized + debuginfo] target(s) in 0.18s 0.

run cargo run to debug the code

& david @ ubuntu in ~/code/rust/first_proj/greeting (master …4) 0 [19:07:19]
~ cargo run
    Finished dev [unoptimized + debuginfo] target(s) in 0.00s
     Running `target/debug/greeting`
Hello, world!

What it's like to use ChatGPT

David Dong — Sat, 04 Feb 2023 19:43:17 +0800

ChatGPT is very hot recently, I also registered an account and experienced it myself. 😉

After registration, I login to ChatGPT’s URL and start testing. I asked a few random professional questions to see how it answered. Here’s how it performed.

❓ Question: How to use DllImport in C#?

ChatGPT: To use DllImport in C#, you need to do the following:

Add a using statement for the System.Runtime.InteropServices namespace at the top of your code file:

using System.Runtime.InteropServices;

Use the DllImport attribute to specify the name of the DLL containing the unmanaged code you want to call and the name of the function within that DLL.

For example:

[DllImport("mydll.dll")] 
private static extern int MyFunction(int x, int y);

Call the unmanaged function from your C# code, just like you would call any other C# method:

int result = MyFunction(10, 20);

The DllImport attribute allows you to call functions in unmanaged DLLs from your C# code. It is used to provide the necessary information for the CLR (Common Language Runtime) to be able to locate and call the function in the DLL at runtime.

❓ Question: Could you introduce the DllImport usage?

ChatGPT: Sure, the DllImport attribute is a way to call functions in unmanaged DLLs (Dynamic Link Libraries) From managed code, such as C#. Unmanaged DLLs are libraries written in languages such as C or C++, which do not run under the .NET runtime. Because of this, you cannot call functions in these DLLs directly From your C# code.

To use DllImport, you need to add a using statement for the System.Runtime.InteropServices namespace at the top of your code file. Then, use the DllImport attribute to specify the name of the DLL containing the unmanaged code you want to call and the name of the function within that DLL.

For example:

using System.Runtime.InteropServices;  
[DllImport("mydll.dll")] 
private static extern int MyFunction(int x, int y);

Once you’ve done this, you can call the unmanaged function From your C# code, just like you would call any other C# method:

int result = MyFunction(10, 20);

The DllImport attribute provides the necessary information for the CLR (Common Language Runtime) to be able to locate and call the function in the DLL at runtime. This allows you to use functions in unmanaged DLLs From your C# code.

Following its answer, I went on to ask…

❓ Question: What does that mean of the parameters EntryPoint and CallingConvention in the calling of DllImport with the C#?

ChatGPT: The EntryPoint parameter of the DllImport attribute is used to specify the name of the function within the DLL that you want to call.

For example:

[DllImport("mydll.dll", EntryPoint = "MyFunction")] 
private static extern 
int MyFunction(int x, int y);

In this case, the EntryPoint is set to "MyFunction", which is the name of the function in the DLL that will be called when MyFunction is called From C# code.

The CallingConvention parameter is used to specify the calling convention of the function being imported. The calling convention determines the order in which parameters are passed to the function, and how the return value is passed back to the caller. The possible values for the CallingConvention parameter are StdCall, Cdecl, and Winapi (the default).

For example:

[DllImport("mydll.dll", EntryPoint = "MyFunction", CallingConvention = CallingConvention.StdCall)] 
private static extern int MyFunction(int x, int y);

In this case, the CallingConvention is set to CallingConvention.StdCall, which specifies that the function uses the standard calling convention. The standard calling convention is the default for most functions in Windows DLLs, so you don’t usually need to specify it explicitly.

The EntryPoint and CallingConvention parameters are optional when using DllImport. If you omit them, the default values will be used. For the EntryPoint parameter, the default value is the name of the method in your C# code, and for the CallingConvention parameter, the default is Winapi. However, it’s a good idea to always specify both of these parameters explicitly to avoid any confusion or potential problems.

And the last question,

❓ Question: What is the CallingConvention.Cdecl and what is the difference with CallingConvention.Cdecl and CallingConvention.StdCall?

ChatGPT: The CallingConvention.Cdecl value specifies that the C calling convention is used for the function being imported. In the C calling convention, parameters are passed on the stack From right to left, and the caller is responsible for cleaning up the stack. This is the default calling convention for functions in C and C++.

The CallingConvention.StdCall value specifies that the standard calling convention is used for the function being imported. In the standard calling convention, parameters are also passed on the stack, but the callee (the function being called) is responsible for cleaning up the stack. This is the default calling convention for most functions in Windows DLLs.

The main difference between these two calling conventions is who is responsible for cleaning up the stack after the function call. In the C calling convention, the caller is responsible for this, while in the standard calling convention, the callee is responsible. This means that if you use the C calling convention and forget to clean up the stack, you will get a stack overflow error. On the other hand, if you use the standard calling convention and forget to clean up the stack, the DLL will crash.

In general, it’s best to use the standard calling convention unless you have a specific reason to use the C calling convention. This is because the standard calling convention is more robust and less error-prone than the C calling convention.

Well done! So far it has performed very well, at least it has answered as precisely as I wanted.

To sum up my own experience, the first impression is “fast, accurate, efficient and powerful”. I feel comfortable using it and unlike traditional search engines, it summarizes and extracts the amount of information available on the internet and presents it to me in a very concise and easy to understand format, which can really improve productivity.

About gerrit.createChangeId

David Dong — Thu, 11 Aug 2022 22:10:09 +0800

git config --bool --get gerrit.createChangeId is a command used in the Git version control system to retrieve the value of the configuration option gerrit.createChangeId. This option determines whether a change-id should be created for each commit message when pushing to a Gerrit server.

The --bool flag specifies that the value being retrieved is a boolean, which means it can be either true or false. The --get flag specifies that the command should retrieve the value of the specified configuration option.

The gerrit.createChangeId option is used in conjunction with Gerrit, which is a web-based code review tool that integrates with Git. When this option is set to true, a change-id will be automatically added to the commit message when pushing to a Gerrit server. This change-id is used by Gerrit to identify a particular change and to track its progress through the code review process.

To set the value of the gerrit.createChangeId option, you can use the command git config --bool --add gerrit.createChangeId true or git config --bool --add gerrit.createChangeId false, depending on the desired value.

Keep old change-id when do git commit

David Dong — Fri, 22 Jul 2022 22:45:56 +0800

When I use git commit --amend command, it will generate a new change-id. But I want to use the previous change-id and don’t want to generate a new one. How should I keep the old change-id and keep only one change-id?

When you use git commit --amend, it creates a new commit that replaces the previous commit. This new commit will have a new commit hash and will be treated as a completely separate commit by Git, even though it may contain the same changes as the previous commit.

To keep the old change-id and have only one change-id, you can use the git commit --amend --no-edit command. This will allow you to amend the previous commit without creating a new commit or changing the commit message. The change-id will remain the same and you will only have one change-id for that commit.

It’s important to note that using git commit --amend can be a powerful tool, but it should be used with caution, as it can cause confusion if used improperly. If you have already pushed the commit to a remote repository, you should be careful when using git commit --amend, as it can cause problems when working with other collaborators. It’s generally a good idea to communicate with your team before using git commit --amend on a commit that has been pushed to a remote repository.

Git hook 配置与使用

David Dong — Fri, 01 Jul 2022 22:10:09 +0800

Hook 就是在执行某个事件之前或之后进行一些其他额外的操作。在 Git 中也有许多的事件（commit、push 等等），每个事件也是对应了有不同的钩子的（如 commit 前，commit 后），那么我们就可以在这些钩子这里配置一些自己需要执行的操作来实现各种各样的需求。

Git Hooks 介绍

Git Hooks 的实现其实非常简单，就是就 .git/hooks 文件下，保存了一些 shell 脚本，然后在对应的钩子中执行这些脚本就行了。比如下图中，这是一个还没有配置 Git Hooks 的仓库，默认会有很多 .sample 结尾的文件，这些都是示例文件。

我们打开 pre-commit.sample 文件看一下其中的内容，大致意思是说这是一个示例，做了一些格式方面的检测，这个脚本默认是不生效的，如果要生效，把文件名改为 pre-commit 即可。

pre-commit 这个钩子是在 git commit 命令执行之前触发。

Git 支持的所有钩子见下表（加粗的为常用钩子）：

PS：完整钩子说明，请参考 @官网链接

Husky 配置

从上面的介绍中，我们知道 Git Hook 保存在 .git 文件夹中。不知你有没有发现这会有一个问题？可能细心的同学已经知道了，Git 是一个多人协作工具，那按理说 Git 仓库中的所有文件都应该被跟踪并且上传至远程仓库的。但是有个例外，.git 文件夹不会，这就导致一个问题，我们在本地配置好 Git Hook 后，怎么分享给其他小伙伴儿呢？copy 吗？那未免太 low 了，都用 Git 了，还 copy，也太不优雅了。这时候，就轮到 Husky 出场了。

Husky 是一个让配置 Git 钩子变得更简单的工具（题外话：Husky 是哈士奇的意思，我猜可能是作者养了条二哈）

下面这些流行的项目都在使用 Husky，可见它确实是一个非常好用的工具：

–no-verify 的使用

git commit --amend --no-verify 命令是一条 Git 命令，它允许你修改最近的提交信息或内容，而不触发 pre-commit 和 commit-msg 钩子。git commit --amend 命令与此类似，但它会在钩子存在的情况下运行这些钩子。这两个命令都会用包含你的修改的新提交来替换之前的提交。

Windows 消息处理机制

David Dong — Sat, 18 Jun 2022 19:43:52 +0800

Windows32 程序是由消息事件驱动的。一个消息，是系统定义的一个 32 位的值，它唯一的定义了一个事件，向 Windows 发出一个通知，告诉应用程序某个事情发生了。

消息队列

Windows 有两种消息队列，系统消息队列和应用程序消息队列。消息在队列中先入先出。
系统消息队列-由系统维护的消息队列，处理系统事件。例如鼠标、键盘等。
程序消息队列-属于每一个应用程序（线程）的消息队列。由应用程序（线程）维护。系统都会为每个 GUI 线程创建一个消息队列，这个消息队列用来存放该应用程序所创建的窗口的消息。
应用程序则通过一个消息循环（消息泵）不断的从消息队列中读取消息，并做出响应。

流程如下

当鼠标、键盘事件被触发后，相应的鼠标或键盘驱动程序就会把这些事件转换成相应的消息，然后发送到系统消息队列，由 Windows 系统去进行处理。
Windows 从系统消息队列中取出一个消息，根据 MSG 消息结构（消息结构体中的窗口句柄 hwnd）确定消息是要被送往那个窗口，然后把取出的消息送往创建窗口的线程的相应队列（Windows 会为每个 GUI 线程创建一个消息队列，该线程中的所有窗体共享这个消息队列）。
线程看到自己的消息队列中有消息，就从队列中取出来，通过操作系统发送到合适的窗口过程去处理。
线程的消息队列的读取和处理消息的部分如下
```
while(GetMessage(&msg,NULL,0,0)){
    if(!TranslateAccelerator(msg.hWnd,hAccelTable,&msg))
      {
          TranslateMessage(&msg);
          DispatchMessage(&msg);
      }
}
```
说明：首先，GetMessage 从进程的主线程的消息队列中获取一个消息并将它复制到 MSG 结构，如果队列中没有消息，则 GetMessage 函数将等待一个消息的到来以后才返回。

注意: GetMessage 会一直等待，直到有接收到消息才返回。

DispatchMessage 函数将把此消息发送给该消息指定的窗口中已设定的回调函数。即用户自定义的处理函数 WndProc。

用户自定义的消息处理函数 WndProc 处理消息。处理消息代码如下。

LRESULT CALLBACK WndProc(HWND hWnd, UINT message, WPARAM wParam, LPARAM lParam){
      switch (message) 
      {
            case WM_COMMAND:
              ...
            case case IDM_EXIT:
              ...
              ...
            default:
              return DefWindowProc(hWnd, message, wParam, lParam);
      }
  return 0
}

说明：用户自定义的窗体处理函数也即 "窗口过程", 窗口过程是一个用于处理所有发送到这个窗口的消息的函数。任何一个窗口类都有一个窗口过程。同一个类的窗口使用同样的窗口过程来响应消息。系统发送消息给窗口过程将消息数据作为参数传递给他，消息到来之后，按照消息类型排序进行处理，其中的参数则用来区分不同的消息，窗口过程使用参数产生合适行为。

消息结构体

typedef struct tagMsg {
    HWND hwnd; //接受该消息的窗口句柄
    UINT message; //消息常量标识符，也就是我们通常所说的消息号
    WPARAM wParam; //32位消息的特定附加信息，确切含义依赖于消息值
    LPARAM lParam; //32位消息的特定附加信息，确切含义依赖于消息值
    DWORD time; //消息创建时的时间
    POINT pt; //消息创建时的鼠标/光标在屏幕坐标系中的位置
}MSG;

发送消息函数 SendMessage () 和 PostMessage () 的区别

SendMessage 这个函数主要是向一个或多个窗口发送一条消息，这个函数是异步函数。一直等到消息被处理之后才会返回。
PostMessage 该函数把一条消息放置到创建 hWnd 窗口的线程的消息队列中，该函数不等消息被处理就马上将控制返回。
被发送的消息（SendMessage）会被立即处理，处理完毕后函数才会返回；被寄送的 (PostMessage) 消息不会被立即处理，他被放到一个先进先出的队列中，一直等到应用程序空线的时候才会被处理，不过函数放置消息后立即返回。

读取消息函数 GetMessage () 和 WaitMessage (), PeekMessage ()

都是读取消息的函数
GetMessage 会一直等到接收到消息才返回。
PeekMessage 函数不会等到有消息放入队列时才返回.
WaitMessage 当一个应用程序无事可做时，该函数就将控制权交给另外的应用程序，同时将该应用程序挂起，直到一个新的消息被放入应用程序的队列之中才返回。

浅谈 Java 线程同步技术

David Dong — Fri, 22 Apr 2022 22:44:19 +0800

Java 里实现线程间的同步有多种方法，比如使用 synchronized 关键字，wait 和 notify，volatile 变量，重入锁，局部变量，阻塞队列或者原子类型变量等。

synchronized

使用 synchronized 关键字：这是最常用的一种方法，它可以修饰代码块或者方法，使得同一时刻只有一个线程能够访问被修饰的代码。例如：

public class SynchronizedDemo {
    public static void main(String[] args) {
        //创建两个线程
        Thread t1 = new Thread(new Runnable() {
            @Override
            public void run() {
                //调用同步方法
                syncMethod();
            }
        }, "t1");

        Thread t2 = new Thread(new Runnable() {
            @Override
            public void run() {
                //调用同步方法
                syncMethod();
            }
        }, "t2");

        //启动两个线程
        t1.start();
        t2.start();
    }

    //定义一个同步方法
    public static synchronized void syncMethod() {
        System.out.println(Thread.currentThread().getName() + "开始执行");
        try {
            Thread.sleep(1000);
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
        System.out.println(Thread.currentThread().getName() + "结束执行");
    }
}

wait 和 notify

使用 wait 和 notify ：这是一种基于对象监视器的机制，它可以让一个线程等待另一个线程的通知，从而实现协作。例如：

public class WaitNotifyDemo {

    private static Object lock = new Object(); //定义一个锁对象

    public static void main(String[] args) {

        //创建两个线程
        Thread t1 = new Thread(new Runnable() {
            @Override
            public void run() {
                synchronized (lock) { //获取锁对象
                    System.out.println("t1开始执行");
                    try {
                        lock.wait(); //释放锁并等待通知
                    } catch (InterruptedException e) {
                        e.printStackTrace();
                    }
                    System.out.println("t1收到通知");
                    System.out.println("t1结束执行");
                }
            }
        }, "t1");

        Thread t2 = new Thread(new Runnable() {
            @Override
            public void run() {
                synchronized (lock) { //获取锁对象
                    System.out.println("t2开始执行");
                    try {
                        Thread.sleep(2000); //模拟耗时操作
                    } catch (InterruptedException e) {
                        e.printStackTrace();
                    }
                    lock.notify(); //唤醒等待的线程
                    System.out.println("t2发送通知");
                    System.out.println("t2结束执行");
                }
            }
        }, "t2");

        //启动两个线程
        t1.start();
        t2.start();

    }

}

volatile

volatile 变量是一种特殊的变量，它可以保证变量在多个线程之间的可见性，也就是说当一个线程修改了 volatile 变量的值后，其他线程能够立即看到最新的值。 volatile 变量还可以防止编译器或者运行时对其进行重排序，从而保证其操作的顺序性。

volatile 变量适合用于以下几种场景：

状态标志：例如一个开关变量，用来控制其他线程的执行流程。单例模式：例如双重检查锁定（double-checked locking）模式，用来保证对象的唯一性和及时性。原子操作：例如 long 或 double 类型的赋值操作，在 Java 中不是原子性的，但是定义为 volatile 后可以保证原子性。

假设有一个 volatile 变量叫做 stop，它用来控制一个线程的执行流程。当 stop 为 false 时，线程继续运行；当 stop 为 true 时，线程停止运行。代码如下:

public class VolatileExample {
    private static volatile boolean stop = false;

    public static void main(String[] args) throws InterruptedException {
        Thread thread = new Thread(new Runnable() {
            @Override
            public void run() {
                int i = 0;
                while (!stop) { // 如果stop为true，跳出循环
                    i++;
                }
                System.out.println("i=" + i);
            }
        });
        thread.start();
        Thread.sleep(1000); // 主线程休眠1秒
        stop = true; // 修改stop的值为true
    }
}

这个例子中，如果不使用 volatile 修饰 stop 变量，那么可能会出现这样的情况：主线程修改了 stop 的值为 true，但是子线程没有看到最新的值，仍然认为 stop 是 false，导致子线程无法停止。这是因为每个线程都有自己的工作内存（缓存），它们会从主内存中复制共享变量的值，并在工作内存中操作共享变量。如果不使用 volatile 修饰共享变量，那么就不能保证其他线程能够及时看到最新的值。

但是如果使用了 volatile 修饰 stop 变量，那么就可以保证当主线程修改了 stop 的值后，其他线程能够立即看到最新的值，并根据最新的值来执行相应的操作。这是因为 volatile 关键字可以强制让编译器每次都从主内存中读取共享变量的值，并且禁止对其进行重排序。

重入锁

重入锁是一种可以被同一个线程多次获取的锁，也就是说当一个线程已经持有了锁，它可以再次请求获取锁而不会被阻塞。重入锁可以避免死锁的发生，也可以提高代码的可读性和可维护性。

Java 中提供了一个重入锁的实现类：ReentrantLock，它实现了 Lock 接口，提供了加锁和释放锁的方法。使用 ReentrantLock 时，需要在 try-finally 块中手动调用 lock () 和 unlock () 方法来保证锁的正确获取和释放。

举个例子吧，假设有一个银行账户类 Account，它有一个余额属性 balance 和一个转账方法 transfer ()，为了保证转账操作的原子性和线程安全性，我们可以使用 ReentrantLock 来对 transfer () 方法进行同步控制。代码如下：

import java.util.concurrent.locks.ReentrantLock;

public class Account {
    private int balance; // 余额
    private final ReentrantLock lock = new ReentrantLock(); // 重入锁

    public Account(int balance) {
        this.balance = balance;
    }

    public void transfer(Account target, int amount) {
        lock.lock(); // 获取锁
        try {
            if (balance >= amount) { // 如果余额足够
                balance -= amount; // 扣除转出金额
                target.balance += amount; // 增加转入金额
            }
        } finally {
            lock.unlock(); // 释放锁
        }
    }
}

这个例子中，如果不使用 ReentrantLock 来对 transfer () 方法进行同步控制，那么可能会出现这样的情况：当两个线程同时对同一个账户对象执行转账操作时，可能会导致余额不正确或者数据不一致的问题。这是因为转账操作涉及到多个共享变量（balance 和 target.balance）的读写操作，并且这些操作不是原子性的。

但是如果使用了 ReentrantLock 来对 transfer () 方法进行同步控制，那么就可以保证当一个线程执行转账操作时，其他线程不能同时执行该操作，并且该操作能够完整地执行而不会被打断。这是因为 ReentrantLock 可以实现互斥访问共享资源，并且保证操作的顺序性和原子性。

原子类型变量

Java 里的原子类型变量是一种可以在多线程环境下保证原子性操作的变量类，它们使用了 CAS（Compare And Swap）算法来实现无锁的并发控制。

原子类型变量有以下几种分类：

标量原子变量类：支持对 int，long 和 boolean 等基本数据类型的操作，例如 AtomicInteger，AtomicLong 和 AtomicBoolean。
原子数组类：支持对 int，long 和引用类型的数组元素进行原子性更新，例如 AtomicIntegerArray，AtomicLongArray 和 AtomicReferenceArray。
原子字段更新程序类：支持使用反射以原子方式更新类的易失性字段，例如 AtomicLongFieldUpdater，AtomicIntegerFieldUpdater 和 AtomicReferenceFieldUpdater。
原子复合变量类：支持对多个变量进行原子性更新，例如 AtomicStampedReference 和 AtomicMarkableReference。下面是一个使用 AtomicInteger 的例子：

import java.util.concurrent.atomic.AtomicInteger;

public class AtomicCounter {
    private AtomicInteger value = new AtomicInteger(0);

    public int increment() {
        return value.incrementAndGet();
    }

    public int decrement() {
        return value.decrementAndGet();
    }

    public int get() {
        return value.get();
    }
}

Git commands and usage tips

David Dong — Tue, 29 Mar 2022 23:17:03 +0800

Git is a powerful version control tool. I would even say it’s one of the best. However, being powerful sometimes means being complex in terms of functionality. Git has a lot of commands. The good thing is that you can do everything you want with them, the bad thing is that you need to remember them. Here, I’ll write down the commands that I’ll use often to keep better track of them and make them easier to search.😉

推送到远程分支

用法：git push <远程主机名> <本地分支名>:<远程分支名>

1. 将本地的 master 分支推送到远程 master 分支

git push origin master

2. 将本地的 dev 分支推送到远程 master 分支

git push origin dev:master

3. 删除远程 master 分支

git push origin :master
git push origin --delete master

4. 添加远程主分支和本地 master 分支的绑定关系

git push -u origin master

绑定后可以直接将当前分支推送到origin主机的对应分支

git push origin

5. 强制推送本地dev分支到远程master分支

git push --force origin dev:master

6. 推送本地当前分支到远程 master分支

git push origin HEAD:master

拉取远程分支到本地

用法：git pull <远程主机名> <远程分支名>:<本地分支名>

1. 拉取远程 master 分支到本地 dev 分支

git pull origin master:dev

2. 拉取远程 master 分支到本地当前分支

git pull origin master

3. 建立本地分支与远程分支的追踪关系

git branch --set-upstream master origin/next

建立追踪后可以直接省略远程分支名

git pull origin

4. git pull origin master:dev 等于

git fetch origin master:dev
git diff dev
git merge dev

5. 采用 rebase 的方式合并本地分支

git pull --rebase origin master:dev

创建新分支

1. 创建 newbranch 分支

git checkout -b newbranch

或者

git branch newbranch
git checkout newbranch

2. 创建新分支并与远程分支关联

git checkout -b tempbarch --track origin/testdevelop

删除分支

1. 删除本地分支

git branch -d newbranch

2. 强制删除本地分支

git branch -D newbranch

3. 删除远程 master 分支

git push origin :master
git push origin --delete master

重命名分支

git branch -m oldbranch newbranch
git branch -M oldbranch newbranch

添加远程分支

git remote add

git remote add origin https://github.com/gangdong/git_learn.git

克隆远程分支

git clone

git clone https://github.com/gangdong/git_learn.git

查看远程提交记录

git log remotes/origin/dev

添加 Github Token 到本地仓库

git remote set-url origin https://<your_token>@github.com/<USERNAME>/<REPO>.git
git remote set-url origin https://ghp_YIKL6G9C5oWxxxxxxssssKsoYg@github.com/gangdong/git_learn.git

git stash

应用场景

当正在dev分支上开发某个项目，这时项目中出现一个bug，需要紧急修复，但是正在开发的内容只是完成一半，还不想提交，这时可以用git stash命令将修改的内容保存至堆栈区，然后顺利切换到hotfix分支进行bug修复，修复完成后，再次切回到dev分支，从堆栈中恢复刚刚保存的内容。
本应该在dev分支开发的内容，却在master上进行了开发，需要重新切回到dev分支上进行开发，可以用git stash将内容保存至堆栈中，切回到dev分支后，再次恢复内容即可。
git stash命令的作用就是将目前还不想提交的但是已经修改的内容进行保存至堆栈中，后续可以在某个分支上恢复出堆栈中的内容。这也就是说，stash中的内容不仅仅可以恢复到原先开发的分支，也可以恢复到其他任意指定的分支上

  git stash
  git stash save "stash 01"
  
  # 恢复
  
  git stash pop 
  
  # 不自动删除堆栈内的内容
  
  git stash apply stash@{n}
  
  # 堆栈中移除某个指定记录
  
  git stash drop
  git stash clear
  git stash list
  
  # 查看某条 stash 的更改
  
  git stash show stash@{n}

合并分支

git merge

1. merge dev 分支和 dev2 分支

git checkout dev
git merge dev2
git mergetool

git checkout dev
git fetch origin dev2
git diff origin/dev2
git merge origin/dev2

2. 手动提交

git checkout dev
git merge --no-commit dev2

3. 如果有冲突，处理冲突

git checkout dev
git merge --no-commit dev2
git ls-files -s
git show :1:filename
git mergetool
git add .
git commit

git cherry-pick

1. 复制 dev2 的 aa3b6ca2 b821c4f7e1c 提交到 dev 分支

git checkout dev
git cherry-pick aa3b6ca2 b821c4f7e1c

注意cherry-pick 多个提交时注意顺序，否则会出现冲突。

git-cherry-pick 只是复制其他分支提交的内容到本地分支，复制后的commit id 和之前 pick的不同。这一点和 git merge/git rebase 有区别。git merge/git rebase 合并的是分支，之前原分支的commit id会带到本地分支。

git rebase

用法：

git rebase   [startpoint]   [endpoint]  --onto  [branchName]
git rebase -i  [startpoint]  [endpoint]

注意：[startpoint] [endpoint]指定了一个编辑区间，如果不指定[endpoint]，则该区间的终点默认是当前分支HEAD所指向的commit (注：该区间指定的是一个前开后闭的区间)。

1. 合并当前分支的多个提交记录

7f66117d96ac6db8661c66c55c5476c9be82622e modify readme.md
6af9edd695f3f51120b888d724eea4d254fe5bb3 master commit 02
3ce944b8cf3b35121936ae0fe59ca7ff76ed81d3 master commit 01
3f901ec367853b7852b30f0023af4398ee48235d my_dev work done.
86624b76ac8a6f16334605e0ee44f64305c4d8c3 my_dev first commit

合并3ce944b8c..7f66117d96ac6 三条提交记录

git rebase -i 3f901ec367853 7f66117d96ac6db

或者

git rebase -i HEAD~3

2. 复制一段提交记录到dev2分支

git rebase -i 3f901ec367853 7f66117d96ac6db onto dev2
git checkout dev2
git reset --hard  HEAD

3. 合并分支

dev 分支rebase master 分支

git checkout dev
git rebase master

4. 冲突处理

如果有冲突，解决掉冲突后，执行

git add .
git rebase --continue

rebase会改变原来的commit ID, 一旦对历史提交做出改变，那么从变基开始的那个提交往后所有的提交的commit ID 都会发生改变。
因此rebase命令使用要遵循以下原则:
1. “从来不要对任何已经提交到公共分支的提交记录做rebase操作，rebase操作会使得公共分支的提交记录变更，导致其他共用该分支的人的代码产生冲突。
2. 不基于rebase分支切换新分支。
3. 不对已经合入远程分支的本地分支做rebase。

git fetch

1. fetch 远程 dev 分支并merge

git fetch origin dev
git diff origin/dev
git merge origin/dev

Git log

1. 显示提交的差异

git log -p

2. 显示 2 个提交信息

git log -2

3. 显示统计信息，包括哪些文件被改动

git log --stat

4. 显示格式，一行显示

git log --pretty=oneline

5. 格式化输出

git log --pretty=format:"%h - %an, %ar-%cr : %s"

按照 哈希值-作者，修改时间-提交时间：提交说明 显示

6. 图形化输出

git log --graph

通常比较有用的是

git log --pretty=formate:"%h - %an, %ar:%s" --graph
git log --pretty=format:"%Cred%h%Creset -%C(yellow)%d%Cblue %s %Cgreen(%cd) %C(cyan)%ar %C(bold blue)<%an>"

7. 按照作者查找提交

git log --author="david.dong"

通常比较常用

git log --author="david.dong" --pretty=oneline

8. 按照内容匹配指定字符串的提交

git log -S

通常比较常用

git log -S "function_name" -p

9. 按照时间范围查找提交

git log --since --until

git log --since="2020-01-01" --until="2020-02-01"
git log --since="3 weeks ago" --until="2 days ago"

10. 限定查找 path/filename 的提交

git log -p -- path/filename

查找 david 对 fpc_bio_interface 的提交

git log --author="david" --pretty=oneline -- my_interface

或者查找 Andy 对 my_db.c 的修改提交记录

git log --author="Andy" -- my_db.c

11. 查找提交说明中包含指定字符串的提交

git log --grep

git log --pretty=oneline --grep="Minor refinement"
git log -p --grep="ISEE"

12. 仅显示merge的提交

git log --merges

13. 不显示merge 提交

git log --no-merges

14. 只打印主线提交

git log --first-parent

15. 只打印某两个提交之间的记录

git log cd50063c9a625..ad3d6ae40d271778ae6

16. 显示某一个分支的提交记录

1. 显示远程 dev 分支记录中有关 README.md 的提交

git log --pretty=oneline origin/dev -- README.md

2. 显示 master 分支记录

git log --pretty=oneline master

17. 查看只在一个分支中的提交

查看只在 master 中的提交记录

git log --pretty=oneline --right-only --no-merges --cherry-pick my_dev...master

查看只在 my_dev 中的提交记录

git log --pretty=oneline --left-only --no-merges --cherry-pick my_dev...master

查看只在 master 有， my_dev 没有的提交记录

git log master ^my_dev --pretty=oneline

或者

git log my_dev..master --pretty=oneline

查看只在 my_dev 有， master 没有的提交记录

git log my_dev ^master --pretty=oneline

或者

git log master..my_dev --pretty=oneline

说明：
... 是列出两个分支不同的提交，等于 master..dev + dev..master, 如果加上 --left-only/--right-only 就等于 ..
.. 是列出一个分支有，另一个分支没有的提交。
因此：
git log --pretty=oneline --right-only --no-merges --cherry-pick my_dev...master
相当于 git log --pretty=oneline master ^my_dev
和 git log --pretty=oneline my_dev..master

18. 不显示 cherry-pick 的提交

git log --cherry-pick

git grep

在某个提交中查找字符串

git grep "ISEE" cd5000634:

git archive

打包某个提交的代码

git archive --format tar.gz --output "linux.5.9.tar.gz" 856deb866d16e29bd659

git show

查看某个提交的内容

git show 856deb866d16e29bd659

重设分支

1. reset –hard

git reset --hard 856deb866

2. reset –soft

git reset --soft 856deb866

区别：reset –hard 执行后之前的提交全部清除，reset –soft 执行后之前的提交会保留到暂存区，相当于staged.

撤消提交

git revert 856deb866

查看全部操作历史

git reflog

生产力工具

David Dong — Tue, 15 Feb 2022 23:11:10 +0800

“工欲善其事，必先利其器”，使用一款好的工具，能帮助我们工作中事半功倍，提升效率。这篇文章记录了我日常使用的一些我认为相当不错的工具和软件。

在线版工具

在生产力工具的选择上我的原则是如果有在线版的工具可以用的话尽量用在线版的，如果有免费的尽量使用免费的，因此除了和工作专业相关的工具外，大部分的工具软件大多数时候都是轻量使用，偶尔或者临时使用一下，没有必要购买许可证或者安装到本地。

使用在线版工具有这么几个优势：

选择多，现在是 2020 年代了，网上的工具实在是太多了，只要你能想到的各种功能，网上都能找到相应的工具，大部分还都有免费的版本可用。不同的网站提供的工具各有特色，可以根据自己的项目需要选择一个最适合的。
轻量级，基于 Web，不用安装一大推软件到本地，占用计算机的资源，拖累计算机的运行速度。
安全，浏览器的运行环境比桌面应用程序更安全，不会有安装软件时强制捆绑安装一些垃圾软件（比如国内某些 xx 全家桶）。也不会有强制弹窗，偷偷修改浏览器配置等问题。

其实目前来说，大多数的软件都有在线版可以使用，性能也不错。所以这篇文章我先以介绍在线版的工具为主，当然还有些工具，比如支持本地快速搜索，网络资源下载工具，专业软件开发调试工具等还是需要运行在本地的，这些我会在下一篇博客里介绍。

这篇文章里提到的工具，大部分是免费使用的，有的是免注册直接就可以使用，有些需要注册后免费使用。针对每个需求，我尽量推荐2个在线工具，可以方便大家选择使用。

在线编译调试

作为一名程序开发人员，首先推荐下面这个网站

网址：https://www.onlinegdb.com/

它可以支持在线编译调试代码，直接可以省掉配置本地开发环境这个环节，支持的编程语言还很多，基本上常见的编程语言都可以在里面找到。特别适合支持一些需要快速验证代码的场合，比如在本地没有安装可开发的环境或者在其他临时使用的计算机上没有可运行的环境，当场安装又来不及，这时可以通过这个网站，直接从你的网上代码仓里远程拉下代码在线调试。是不是很方便？

另外下面这个在线调试网站 ideone 也可以使用，亲测感觉编译的速度比上面的网站要稍微慢一些。

网址：https://ideone.com/

GITHUB 资源查找

程序员会经常需要在Github上查找有用的项目，推荐使用 gitlogs，可以在 GitHub 上方便的查找到你感兴趣的项目。

网址：https://www.gitlogs.com/

流程图工具

我推荐 draw.io,

网址：https://app.diagrams.net/

理由是

功能强大，不仅仅能画流程图，还包括其他各种图表。
操作简单，不需要花费很多时间学习就可以快速的完成绘制。
易用性好，提供了很多模板，很方便使用。

综合工具箱

有的时候工作中会用到一些不常见的工具，下面的这两个网站提供了各种各样的小工具，基本上能够涵盖了大多数的需要了。

网址：

功能可以说很齐全，比如包括：

字符串编解码工具
进制转换工具
Jason/XML 格式转换
代码比对
随机数生成器
MD5/AES 加密
正则表达式
ascii 码编辑器
Unix 时间戳转换

等等。

以上的网站相当于是一个各类工具的合集，如果有此类需要的时候可以去里面找一下，说不定就能找到适合的工具。

OCR 工具

格式转换

Online converter 提供了各种转换工具，包括

视频转换
音频转换
PDF 转换
Office 文档格式转换等

是一个综合的转换工具集合。

免费用户可以

每天转换次数	每次转换文件数量	每个转换的文件大小
20	5	100MB

通常对应一般的轻量化的需求，都可以满足。

网址：https://www.online-convert.com/

PDF 转换

针对 PDF 转换，推荐 smallpdf 和 speedpdf

网址：

这两个网站提供了丰富的 pdf 处理能力，易于操作，其中包括：

合并PDF/拆分PDF/压缩PDF
PDF转换至Word/PowerPoint/Excel
Word/PowerPoint/Excel 转换至PDF文件
PDF转JPG、JPG转PDF
页码、水印、旋转PDF
HTML转换PDF
PDF解密、PDF加密
排版PDF文件、修复PDF 等

免费用户也完全可以使用。

图片压缩

Squoosh 是谷歌推出的一款在线图像压缩工具，可帮助网站开发人员快速压缩图片，在保持图片质量的同时，提升网站访问速度。

网址：https://squoosh.app/

TinyPNG 也是可以的。

TinyPNG – Compress WebP, PNG and JPEG images intelligently

图片编辑处理

PhotoKit.com 是免费的在线 P 图编辑器，可以完成在线抠图、改图、修图、美图等。方便易用。一键轻松抠图、改图、修图和美图应用，包含图片裁剪、调整大小、旋转等基本编辑。对于轻量级的编辑需求完全可以满足，没有必要去用 photoshop 等专业性很强的软件。

免费用户可以使用所以的处理工具，导出时需要额外的等待时间。

网址：PhotoKit.com

另外如果专门针对去水印的话也可以使用 Inpaint ，可以很方便的一键去除去除图片上的水印。不需要任何技术门槛。

网址：https://theinpaint.com/

网络素材

下面几个网站提供了大量精美的图片，有需要贴图找图的时候可以去浏览一下。

网址：

壁纸

专门提供免费壁纸的网站，从里面可以下载大量精美的壁纸。

网址：Wallpaper Abyss - HD Wallpapers, Background Images (alphacoders.com)

Emoji 表情，ICON 图标

这两个网站提供了大量的在线图标和 Emoji，可以免费下载使用。

Icons - Iconfinder.com

Download Emoji Images For Free! - emojiisland.com

文档搜索

推荐鸠摩搜索，鸠摩搜索是一个文档类搜索的引擎，本身不存储文档，通过 AI 算法检索网上第三方资源，自动生成到第三方网站的链接。

Jiumo Search 鸠摩搜索 - 文档搜索引擎 (jiumodiary.com)

在线制作图标，Logo，艺术字

艺术字体在线生成器艺术字转换器字体转换器免费书法字体设计大全-找字体网 (qt86.com)

免费logo在线制作-字体logo-logo设计-U钙网 (uugai.com)

在线翻译

除了大家都知道的 Google Translate 外，我还常用 DeepL，号称全球最准确的翻译。翻译出来的质量确实不错，也可以直接翻译整篇文章，不过免费用户在字数和文章数量上有限制，如果不是大量的文章需要翻译的话，使用起来还是不错的。

Edge 插件

最后再推荐几个 Edge 浏览器的插件：

AdGuard - 广告拦截器，当前的网络环境下，相信每一个上网的人都会安装一款广告拦截程序，AdGuard 号称世界上最好的广告拦截程序，网上评价很高，除了屏蔽广告，AdGuard 还提供了许多其他工具，其中最重要的是网络钓鱼筛选器。AdGuard 可以很方便的通过 Edge 插件市场直接安装。
oneTag - 当我们同时打开很多网页的时候，只需要点一下 OneTag 按钮，即可把 Tag 页变成列表的形式，当需要再次访问这些标签页时，可以单独或全部恢复它们。通过简书浏览器中打开的标签数量，OneTag 可以大量的节省内存。
TamperMonkey - 大名鼎鼎的“油猴”，相信很多人听说过或者正在用，它可以让你再网页上执行任何的脚本，总之装上它就对了。
沙拉查词 - 沙拉查词是浏览器插件形式的翻译软件，功能强大，最值得一提的是它在使用的设计上非常人性化，充分考虑了人们阅读的习惯，让人阅读时不会感受到它的存在，在需要翻译时又能瞬间调出，轻松快速的完成翻译。是不可多得的一款翻译软件。
Grammarly for Microsoft Edge - 语法错误检查和标注插件。
IDM integration Module - IDM 大家一定都听说过了，这款插件需要配合 IDM desktop 软件使用，好处是在安装完 IDM desktop, 第一次运行时，软件会自动安装插件到浏览器，从此就可以用 IDM 从网上直接下载资源了。😜

Desktop 工具

本地文件搜索

我目前在用的是 Everything + AnyTxt Searcher, Everything 用于文件搜索，AnyTxt Searcher用于字符串的搜索。这两个工具和 Linux 下的 find + grep的作用类似，区别是前者是带有 GUI 工具，而 find + grep是命令行工具。find + grep 也有针对 Windows 平台的 .exe 版本，我有时在终端下也会使用，体验上和在 Linux 下没有区别。另外在 Linux 下还会使用 fsearch 这个工具。

快速启动和文件搜索

用过一段时间的 WOX和 uTools，最后还是换回了 Listary, 快速启动之类的功能都基本类似，Listary最大的优点是可以自动记录并定位到上次资源管理器打开的路径，这在 win 里简直是个神器。

工具插件聚合

我目前用的是 quicker, 原因无它，quicker提供了海量的动作库，一键式添加。比如 OCR，文件处理，批量重命名，大小写转换，使用录屏功能自动输入密码等。而且针对不同的软件，有专有的动作库，比如 EXCEL 的处理等，使用好了可以大大的提高效率。

编程文档库

Velocity 配合 zeal 一起使用。

人工智能辅助

ChatGPT：一个由 OpenAI 发布的 ChatGPT，知识检索、创作文章、编写代码、聊天、AI 绘图。链接：ChatGPT (openai.com)
cht.sh：增强 cheat 和 tldr，提供编程建议，语法解释。
Copilot: AI 辅助编程插件。链接： GitHub Copilot - GitHub Docs

设备互传

坚果云
Logitech Flow
LANDrop：多台电脑，手机之间通过 LANDrop 互传文件。需要设备上安装 LANDrop 应用。
SnapDrop：局域网云共享，同一个局域网内多台设备通过登录 snapdrop 网站点对点传送文件，无需安装应用。网址：https://snapdrop.net/
奶牛快传：https://cowtransfer.com/

流程图

drawio
坚果云轻应用

拼图

ShareX

长截图

faststone capture
ShareX

Markdown 编辑

obsidian
marktext
坚果云轻应用

之前用的比较多的是 typroa，自从它变成收费的后，就改用 marktext替代。typroa和 marktext都有一个缺点，那就是对于单个 markdown 文件的编辑都没得说，但是在文件的组织管理和查找上都功能缺失。直到最近体验了 obsidian 以后，完美的弥补了上述的缺点，而且还提供了大量的插件可以增强功能，简直是 perfect，现在已经完全转到 obsidian来了。

笔记软件

obsidian

文件恢复

Recuva

大文件查找清理

WizTree64

一个字，快。

Git GUI

sublime merge (Linux)
source tree
gitkraken
GitHub desktop
tig
gitui

Git GUI 的工具有很多，但是平时用的最多的还是命令行，😊！偶尔需要用 GUI 的时候，Linux 下用 sublime merge多一些，Windows 下是 Github Desktop 和 source tree。

文档比对

meld
beyond compare

Linux 下使用 meld, Windows 下使用 beyond compare。

磁盘划分

gparted (Linux)

Linux 下的工具。

文件预览

quicklook
gnome-sushi (Linux)

剪切板增强

ditto
copyq (Linux)

终端

cmder
windows terminal
tabby
mobaXterm (远程 SSH)
terminator (Linux)
gnome terminal (Linux)

终端的选择太多了，目前主力在用的是 cmder + terminator。

多媒体

potplayer
honeyview

potplayer看视频， honeyview看图。这两个软件都是小巧，功能多，占用资源少，运行快。

计算机管理

ccleaner
geek

文本编辑器

EmEditor
notepad++
Notepad 3
geany (Linux)
gedit

Windows 下推荐 EmEditor，超大文件秒开，搜索功能使用起来很方便。Linux 下 geany非常好用。

IDE & 代码编辑器

source insight
vscode
idea
pycharm
android studio
visual studio 2022
sublime

这个就不多说了，程序员对以上的软件都很熟悉，针对不同的编程语言，每个人都会有自己喜爱的编辑器和 IDE。萝卜青菜，各有所爱吧。

密码管理

keepass

优点是密码本地存储，安全，开源免费，功能强大，有插件支持。

笔记管理

obsidian
onenote
masscode (Linux)

之前使用 onenote, 也十分好用，现在转到 obsidian，原因有二：

onenote 的文档不是通用的格式
onenote 不支持 markdown 语言，需要安装 onemark插件

以上两点 obsidan都完美解决，本地存储的 markdown 文件。任何文本编辑器都可以浏览编辑。

截图 & 录屏

Snipaste
Captura
ScreenToGif
ShareX

Snipaste 截图用，Captura 录屏工具。

虚拟机

VMware
VirtualBox

这个就不多说了，二者二选一个即可。

下载工具

区别是 IDM 收费软件，NDM 免费使用，速度上差不多。

文件管理

XYplorer
double-commander (Linux)
groupy
windowTop
DropIt
Folder Painter
folder-color （Linux）

Windows 下的文件管理软件有很多，比如有名的 Total Commander, Free Commander, Directory Opus等，XYplorer 也是其中之一，不多说。Linux 下就推荐 double commander,功能和 total commander类似，因为 total commander 没有 Linux 的版本。剩下的几款都是增强型软件，比如增加文件浏览器的 TAB 页，窗口置顶，文件夹着色等等。

文件格式转换

pandoc

文档格式转换利器，命令行使用，功能强大。

文件同步

freefilesync

压缩/解压缩

7zip
peazip

博客

jekyll

从 MarkdownPad2 到 Typora - 使用 Typora 的直观感受

David Dong — Fri, 11 Feb 2022 09:24:16 +0800

一直以来我都是用 MarkdownPad2 撰写自己的博客文章，也没有觉得哪里不好。关注到 typora 这个软件是因为最近网上讨论的比较热烈的这个软件要收费的消息，于是上网 Google 了一下这个软件才发现很多人对这个软件的评价很高，这就引起了我的兴趣，于是我决定下载这个软件亲自试用一下。😉

第一眼感觉

下载安装后，打开这款文本编辑器，第一眼感觉着实有些惊艳到我，整个 UI 给人的感觉是 “干净，简洁”，没有任何多余的东西。这个设计的 style 还是比较符合我的审美。对比一下 MarkdownPad2 感受会更明显。

MarkdownPad2 看上去就是比较传统的 UI 设计，典型的 windows 窗口的布局，编辑区，工具栏，菜单… 和其他的 windows 软件没有多大的区别，当然好处是这样比较符合大多数人的使用习惯，但是也使人提不起多大的使用兴趣。

文本输入

不得不说 typora 在文本的输入方式上和其他的编辑软件还是有显著不同，这个不同就是 typora 采用实时预览，也就是【所见即所得】，不像其他的 Markdown 编辑器那样分栏显示，分为编辑区和预览区，占用屏幕空间。typora 甚至连 Markdown 语法的标记都在实时预览中消去了。当你离开正在编辑的有格式的文本段后，Typora 会自动隐藏 Markdown 标记。一开始还有些不太习惯，但是简简单单的敲入字符，编写完几行内容后，几分钟的时间我就彻底喜欢上这种方式，感觉再也回不去过去那种模式。

typora 对 Markdown 语法的支持也非常的好，我用它敲写 Markdown 语句完全没有违和感。感觉从 MarkdownPad2 切换到 typora 几乎没有花费多少时间成本，除了前几分钟用来适应【所见即所得】以外，其他的都非常顺畅。

typora 还提供了【打字机模式】和【专注模式】这两种独有的编辑模式，用过以后效果非常不错！

功能性

功能性上 typora 也有其独到的地方，比如 typora 可以很方便的把 markdown 文件导出为其他文件格式，比如 pdf, 图像格式(.jpg, .png)，甚至在 pandoc 的帮助下生成更多的格式，像 docx, Latex, Epub 都是可以的。这点我在 MarkdownPad2 上是没有看到。

另外 typora 还支持很多的快捷键，常用操作用都可以用快捷键完成，可以大大提高效率。

另外 typora 还有其他一些人性化的特性，比如快速插入 Emoji 图标，很方便的使用 Latex 编写数学公式，还可以编写流程图，也支持代码块高亮，Typora 原生支持许多编程语言代码块的语法高亮，基本日常常用的编程语言它都能很好地支持。有了这些特性，就可以像写代码一样编写文档了，简直是程序猿们的最爱啊~~

扩展性

typora 有很多主题可以选择，如果你不喜欢现在的 UI，可以很方便的从网上找到一款喜欢的主题，瞬间完成自己想要的风格。不像 MarkdownPad2，百年不变的外观，没有可玩性。

以上是我使用下来最直观的感受，因为使用时间不长，有很多功能应该说还有待发掘。

其他

关于 typora, 用 typora 官方对自己的介绍来说就是：

简洁美观
实时预览
扩展语法
跨平台
免费

实际上我自己使用下来的感受也是差不多的，跨平台的特性我还没有在其他平台上使用过，有机会会在其他系统平台用一下。

这款软件之前是完全免费使用的，最近已经开始收费了，价格倒是不贵，根据官网的消息，目前Typora的收费方式为买断式消费，价格为 89元 RMB，最多可以在 3 台设备上使用。

好消息是目前的收费只针对不久前正式发布的 V1.0 版本，之前的发布的 beta 版本还可以继续免费使用。目前我自己在用的 0.11.18 也还是免费使用的。

写在最后

总结一下我自己对这款编辑器的体会就是 “快速，简洁，外观漂亮和功能强大”，它【极简】的设计，【实时预览】的特性和对 Markdown 良好的支持，让我感觉用它来写作非常舒服，细枝末节处的人性化增加了写作的欲望，用现在比较流行的说法就是提高了生产力。

最后贴一张和 MarkdownPad2 对比图，看起来 MarkdownPad2 就显得有那么一点笨重和陈旧了。typora 符合了我对文本编辑器的大多数的需求，后面我会切换到 typora 来作为文本编辑器使用，MarkdownPad2 可以卸载掉了~~

Securing Connections with SSH - A Comprehensive Guide

David Dong — Wed, 15 Dec 2021 20:11:11 +0800

Introduction

Secure Shell (SSH) is a cryptographic network protocol widely used for secure communication over an unsecured network. I used it almost everyday, this article will delve into the encryption principles and applications of SSH, followed by a guide on installing and utilizing SSH on Linux systems, and finally, exploring methods to enable passwordless login for remote hosts.

Chapter 1: Encryption Principles and Applications of SSH

1.1 Symmetric Encryption:

SSH utilizes symmetric encryption algorithms like AES (Advanced Encryption Standard) to encrypt data during transmission.
Symmetric encryption ensures that the data exchanged between the client and server remains confidential and cannot be intercepted by malicious actors.

1.2 Asymmetric Encryption:

Asymmetric encryption, based on public-private key pairs, is employed for authentication and key exchange.
SSH uses algorithms like RSA or ECDSA to generate key pairs, with the client possessing the public key and the server holding the private key.
This asymmetric encryption ensures secure authentication and protection against man-in-the-middle attacks.

1.3 Use Cases of SSH:

Remote login: SSH enables users to securely log in to remote systems over the network.
Secure file transfer: It facilitates the secure transfer of files between systems using utilities like SCP (Secure Copy) or SFTP (SSH File Transfer Protocol).
Tunneling: SSH tunneling allows for secure communication between two devices by encapsulating data within SSH connections, useful for accessing services behind firewalls or bypassing censorship.

Chapter 2: Installing and Using SSH on Linux

2.1 Installation:

Most Linux distributions come pre-installed with SSH packages. However, if not available, SSH can be easily installed using package managers like apt, yum, or dnf.

For example, on Debian-based systems:

sudo apt update
sudo atp install openssh-server
sudo apt install openssh-client

check SSH status
```
sudo systemctl status ssh
```
start SSH service
```
sudo systemctl start ssh
```

stop SSH service

sudo systemctl stop ssh
sudo systemctl disable ssh

2.2 Configuration:

SSH configuration files are located in /etc/ssh/ directory.
Users can customize SSH settings such as port number, authentication methods, and access control through the sshd_config file.
```
ssh username@hostname
```
Replace ‘username’ with the remote user’s username and ‘hostname’ with the IP address or domain name of the remote server.

3.1 Generating SSH Key Pair:

To enable passwordless login, first, generate an SSH key pair on the local machine using the ssh-keygen command.

ssh-keygen -t rsa -b 4096 -C "username@user.com"

This command will create a public key (id_rsa.pub) and a private key (id_rsa) in the ~/.ssh/ directory.

3.2 Copying Public Key to Remote Host:

Next, copy the public key to the ~/.ssh/authorized_keys file on the remote host using the ssh-copy-id command or manually.
```
ssh-copy-id username@hostname
```

Once the public key is added to the authorized_keys file on the remote host, SSH will authenticate the user using the private key without requiring a password.
Users can now log in to the remote host securely without entering a password.

A common example is to use this method to push commits to github repository, the workflow is similiar.

ssh-keygen -t rsa -b 4096 -C "xxx@xxx.com" 
#write the private key to ssh agent
ssh-add ~/.ssh/id_rsa
# then login the github accounts,  
# add the public key id_rsa_pub into github ->settings->SSH and GPG keys
# check the connecting status
ssh -T git@github.com
# check the local branch status,
# if binding to remote branch
git branch -vv
# looking for the remote url information.
git remote -v
# if the url is using https format, switch to ssh link
# the link informaion can be found the github repository page
git remote set-url origin git@github.com:username/repository_name.git
# push to remote without inputing password
git push origin 

3.4 Known Host

known_hosts is a text file that stores the host keys of remote servers your computer has connected to in the past. Each time you connect to a new remote server, the SSH client will ask you to confirm the host key and then save it to the ~/.ssh/known_hosts file.

The purpose of this file is to help detect man-in-the-middle attacks. If the host key of a server you’ve previously connected to changes (for example, due to the server being reinstalled or tampered with by a man-in-the-middle attacker), the SSH client will issue a warning, indicating a potential security risk.

However, there are situations where you might encounter issues, such as when the host key changes for legitimate reasons or when you switch between different connection methods (e.g., connecting via IP address versus domain name). In such cases, you may need to manually edit the ~/.ssh/known_hosts file to remove or update the old host key.

In summary, ~/.ssh/known_hosts is a file that stores the known host keys for SSH clients, helping ensure that the remote servers you connect to are the ones you expect, and providing a level of security against unauthorized access

Conclusion

SSH stands as a cornerstone of secure communication, offering robust encryption and authentication mechanisms for remote access and data transfer. Understanding its encryption principles, installation, and utilization on Linux systems, along with implementing advanced features like passwordless login, empowers users to establish and maintain secure connections in today’s interconnected digital landscape.

C# 的 IntPtr 类型

David Dong — Fri, 10 Dec 2021 20:29:33 +0800

C# 中，IntPtr 是一个代表内存位置指针的类型。它被用来存储一个变量或一个对象在内存中的地址。IntPtr 是一个整数类型，但它的大小与平台有关。在 32 位系统中，IntPtr 的大小为 32 比特（4字节），在 64 位系统中，它的大小为 64 比特（8字节）。

IntPtr 通常在处理非托管代码或与其他使用指针的语言相互操作时使用。例如，如果你从动态链接库（DLL）中调用一个以指针为参数的函数，你可以使用IntPtr将一个变量的地址传递给该函数。C# 中主要用它调用 C++\C 封装的 DLL 库；

下面主要介绍 IntPtr 的常见用法。

1 .int 类型与 IntPtr 类型之间的转换

using System;
using System.Collections.Generic;
using System.Linq;
using System.Runtime.InteropServices;
using System.Text;
using System.Threading.Tasks;

namespace MyIntPtr
{
    class Program
    {
        static void Main(string[] args)
        {
            int nValue1 = 10;
            int nValue2 = 20;
            //AllocHGlobal(int cb):通过使用指定的字节数，从进程的非托管内存中分配内存。
            IntPtr ptr1 = Marshal.AllocHGlobal(sizeof(int));
            IntPtr ptr2 = Marshal.AllocHGlobal(sizeof(int));
            //WriteInt32(IntPtr ptr, int val):将 32 位有符号整数值写入非托管内存。
            //int->IntPtr
            Marshal.WriteInt32(ptr1, nValue1);
            Marshal.WriteInt32(ptr2, nValue2);
            // ReadInt32(IntPtr ptr, int ofs):从非托管内存按给定的偏移量读取一个 32 位带符号整数
            //IntPtr->int
            int nVal1 = Marshal.ReadInt32(ptr1, 0);
            int nVal2 = Marshal.ReadInt32(ptr2, 0);
            //FreeHGlobal(IntPtr hglobal):释放以前从进程的非托管内存中分配的内存。
            Marshal.FreeHGlobal(ptr1);
            Marshal.FreeHGlobal(ptr2);
        }
    }
}

2.string 类型与 IntPtr 之间的转换

using System;
using System.Collections.Generic;
using System.Linq;
using System.Runtime.InteropServices;
using System.Text;
using System.Threading.Tasks;

namespace MyIntPtr
{
    class Program
    {
        static void Main(string[] args)
        {
            string str = "aa";
            IntPtr strPtr = Marshal.StringToHGlobalAnsi(str);
            string ss = Marshal.PtrToStringAnsi(strPtr);
            Marshal.FreeHGlobal(strPtr);  
        }
    }
}

3. 结构体与 IntPtr 之间的转换

using System;
using System.Collections.Generic;
using System.Linq;
using System.Runtime.InteropServices;
using System.Text;
using System.Threading.Tasks;

namespace MyIntPtr
{
    class Program
    {
        public struct stuInfo
        {
            public string Name;
            public string Gender;
            public int Age;
            public int Height;
        }
        static void Main(string[] args)
        {
            stuInfo stu = new stuInfo()
            {
                Name = "张三",
                Gender = "男",
                Age = 23,
                Height = 172,
            };

            //获取结构体占用空间的大小
            int nSize = Marshal.SizeOf(stu);
            //声明一个相同大小的内存空间
            IntPtr intPtr = Marshal.AllocHGlobal(nSize);
            //IntPtr->Struct
            Marshal.StructureToPtr(stu, intPtr,true);
            //Struct->IntPtr
            stuInfo Info =(stuInfo)Marshal.PtrToStructure(intPtr, typeof(stuInfo));

            Console.ReadKey();

        }
    }
}

4. 调用 C++\C 封装的 DLL 库

Copy codeusing System;
using System.Runtime.InteropServices;

namespace IntPtrExample
{
    class Program
    {
        // Import the DLL function that takes an IntPtr argument
        [DllImport("mylibrary.dll")]
        static extern void MyFunction(IntPtr ptr);

        static void Main(string[] args)
        {
            // Create an integer variable
            int x = 10;

            // Allocate memory for the variable and get a pointer to it
            IntPtr ptr = Marshal.AllocHGlobal(Marshal.SizeOf(x));
            Marshal.StructureToPtr(x, ptr, false);

            // Call the DLL function and pass the pointer to the variable
            MyFunction(ptr);

            // Free the allocated memory
            Marshal.FreeHGlobal(ptr);
        }
    }
}

在这个例子中，MyFunction() 函数从一个DLL中导入，并接受一个 IntPtr 参数。Main() 方法使用 Marshal.AllocHGlobal() 方法为一个整数变量分配内存，然后使用一个 IntPtr 变量将该变量的地址传递给 MyFunction()。

注意，当使用 IntPtr 时，正确管理被分配的内存是很重要的。在上面的例子中，内存是用 Marshal.AllocHGlobal() 分配的，然后在不再需要时用 Marshal.FreeHGlobal() 释放。未能正确管理内存会导致内存泄漏或其他问题。

浅析 C# 中的线程同步

David Dong — Sat, 11 Sep 2021 22:14:11 +0800

线程间同步是指在多线程环境下，保证共享资源的安全和一致性的机制。 C# 中提供了多种方式实现线程间同步。例如：

lock 语句：使用一个对象作为锁，保证一次只有一个线程可以进入临界区；
Interlocked 类：提供了原子操作，如递增、递减、交换和读取值；
Monitor 类：提供了锁定对象、等待信号和通知信号的方法；
Mutex 类：提供了跨进程的互斥锁，可以用来同步不同进程中的线程；
Semaphore 类：提供了一个计数器，限制同时访问共享资源的线程数；
AutoResetEvent 和 ManualResetEvent 类：提供了信号量，可以用来通知或等待其他线程的状态变化；

lock 语句

C# 中的 lock 语句是用来同步多个线程对共享资源的访问的。lock 语句需要一个对象作为锁，当一个线程进入 lock 语句块时，它会获取该对象的互斥锁，执行语句块，然后释放锁。在持有锁的期间，其他线程无法获取该锁，只能等待释放。

lock 语句的一般用法如下：

using System;
using System.Threading;

class Program
{
    static int counter = 0;
    static object locker = new object();

    static void Main(string[] args)
    {
        Thread t1 = new Thread(Increment);
        Thread t2 = new Thread(Increment);

        t1.Start();
        t2.Start();

        t1.Join();
        t2.Join();

        Console.WriteLine("Counter: " + counter);
    }

    static void Increment()
    {
        for (int i = 0; i < 100000; i++)
        {
            lock (locker)
            {
                counter++;
            }
        }
    }
}

interlocked 类

C# 中的 Interlocked 类是用来提供原子操作的，即一次只能有一个线程执行的操作。Interlocked 类可以保证对共享变量的读取、修改和写入是线程安全的。 Interlocked 类提供了一些静态方法，如：

Increment：将一个整数变量递增并返回新值。
Decrement：将一个整数变量递减并返回新值。
Add：将一个整数变量增加指定的值并返回新值。
Exchange：将两个变量交换并返回原始值。
CompareExchange：比较两个变量是否相等，如果相等则交换它们，并返回原始值。

下面是一个使用 Interlocked 类实现线程间同步的例子：

using System;
using System.Threading;

class Program
{
    static int counter = 0;

    static void Main(string[] args)
    {
        Thread t1 = new Thread(Increment);
        Thread t2 = new Thread(Increment);

        t1.Start();
        t2.Start();

        t1.Join();
        t2.Join();

        Console.WriteLine("Counter: " + counter);
    }

    static void Increment()
    {
        for (int i = 0; i < 100000; i++)
        {
            Interlocked.Increment(ref counter);
        }
    }
}

Monitor 类

Monitor 类是 C# 中用于提供多线程环境下的线程安全的一个类。它可以确保一次只有一个线程能够访问临界区代码，避免线程之间的竞争条件。Monitor 类包含了一些静态方法，它们操作一个控制对临界区访问的对象。Monitor.Enter 和 Monitor.Exit 方法用于锁定和释放对象或资源。这里有一个简单的例子，它演示了如何使用 Monitor 类来保护一个共享资源（一个计数器）免受并发访问的影响。请注意，我们需要在 try 和 finally 块中使用 Monitor.Enter 和 Monitor.Exit 方法。

using System;
using System.Threading;

namespace MonitorExample
{
    class Program
    {
        static int counter = 0; //shared resource
        static object lockObject = new object(); //object to lock

        static void Main(string[] args)
        {
            Thread t1 = new Thread(IncrementCounter);
            Thread t2 = new Thread(IncrementCounter);

            t1.Start();
            t2.Start();

            t1.Join();
            t2.Join();

            Console.WriteLine("Final Counter Value: " + counter);
        }

        static void IncrementCounter()
        {
            bool isLockTaken = false;
            try
            {
                Monitor.Enter(lockObject, ref isLockTaken); //lock
                for (int i = 0; i < 100000; i++)
                {
                    counter++;
                }
            }
            finally
            {
                if (isLockTaken)
                    Monitor.Exit(lockObject); //release
            }
        }
    }
}

这个例子程序的目的是演示如何使用 Monitor 类来保护一个共享资源（一个计数器）免受并发访问的影响。程序中创建了两个线程 t1 和 t2，它们都调用了同一个方法 IncrementCounter，该方法用于对计数器进行递增操作。为了避免两个线程同时修改计数器的值，造成数据不一致的问题，我们需要使用 Monitor 类来锁定一个对象（lockObject），这样一次只有一个线程能够进入临界区（for 循环）。在 try 块中，我们使用 Monitor.Enter 方法来尝试获取锁，并将 isLockTaken 参数设置为 true。如果获取成功，我们就可以对计数器进行递增操作。在 finally 块中，我们检查 isLockTaken 参数是否为 true，如果是，则说明我们已经获取了锁，那么我们就需要使用 Monitor.Exit 方法来释放锁，以便其他线程可以访问临界区。最后，在主线程中，我们等待两个子线程执行完毕，并打印出最终的计数器值。

[参考]：

Monitor.Enter () 方法是用于锁定一个对象或资源，以便一次只有一个线程能够访问临界区。它有两个重载版本，一个只接受一个 object 参数，另一个接受一个 object 参数和一个 ref bool 参数。第一个版本会尝试获取锁，如果成功，则返回，如果失败，则阻塞当前线程，直到获取锁为止。第二个版本会尝试获取锁，并将 ref bool 参数设置为 true 或 false，表示是否获取成功。这个版本可以指定超时时间或取消令牌。

[参考]：

Monitor.Enter Method (System.Threading) | Microsoft Learn

Mutex 类

Mutex 类是一个同步原语，用于在多线程环境中保护共享资源的访问。Mutex 类提供了 WaitOne 和 ReleaseMutex 方法，分别用于请求和释放互斥锁的所有权。Mutex 类具有线程关联性，即只有拥有互斥锁的线程才能释放它。

一个简单的例子是，假设有两个线程 A 和 B，需要对一个共享变量 count 进行增加操作。为了避免数据竞争，可以使用一个 Mutex 对象来同步对 count 的访问，如下所示：

using System;
using System.Threading;

public class MySharedMutexCounter
{
    public static int count = 0;
    public static Mutex ObjMutex = new Mutex();
}

public class MyThreadA
{
    public Thread Thrd;

    public MyThreadA(string name)
    {
        Thrd = new Thread(this.Run);
        Thrd.Name = name;
        Thrd.Start();
    }

    void Run()
    {
        for (int i = 0; i < 5; i++)
        {
            // Request ownership of the mutex.
            MySharedMutexCounter.ObjMutex.WaitOne();

            // Increment the shared variable.
            Console.WriteLine(Thrd.Name + " is incrementing count.");
            MySharedMutexCounter.count++;
            Console.WriteLine("Count is now " + MySharedMutexCounter.count);

            // Release ownership of the mutex.
            MySharedMutexCounter.ObjMutex.ReleaseMutex();
        }
    }
}

public class MyThreadB
{
    public Thread Thrd;

    public MyThreadB(string name)
    {
        Thrd = new Thread(this.Run);
        Thrd.Name = name;
        Thrd.Start();
    }

    void Run()
    {
        for (int i = 0; i < 5; i++)
        {
            // Request ownership of the mutex.
            MySharedMutexCounter.ObjMutex.WaitOne();

            // Increment the shared variable.
            Console.WriteLine(Thrd.Name + " is incrementing count.");
            MySharedMutexCounter.count++;
            Console.WriteLine("Count is now " + MySharedMutexCounter.count);

            // Release ownership of the mutex.
            MySharedMutexCounter.ObjMutex.ReleaseMutex();
        }
    }
}

class Program
{
   static void Main()
   {
      Console.WriteLine("Main thread starting.");

      // Construct two threads.
      var mt1 = new MyThreadA("Child #1");
      var mt2 = new MyThreadB("Child #2");

      mt1.Thrd.Join();
      mt2.Thrd.Join();

      Console.WriteLine("Main thread ending.");
   }
}

[参考]：

Semaphore 类

Semaphore 类是一个同步原语，用于控制对一组资源的并发访问。Semaphore 类可以表示一个命名的（系统范围的）或本地的信号量。它是对 Win32 信号量对象的一个薄封装。Win32 信号量是计数信号量，可以用于控制对一个资源池的访问。

Semaphore 类提供了构造函数，用于指定初始和最大的并发访问数，以及可选的系统信号量对象的名称。它还提供了 WaitOne 和 Release 方法，分别用于请求和释放信号量的计数。

一个简单的例子是，假设有三个线程 A、B、C，需要对三个共享资源 X、Y、Z 进行操作。为了避免同时有多个线程操作同一个资源，可以使用一个 Semaphore 对象来限制最多只能有三个线程同时访问资源池，如下所示：

using System;
using System.Threading;

public class MySharedResource
{
    public static string[] resources = new string[] { "X", "Y", "Z" };
    public static Semaphore ObjSemaphore = new Semaphore(3, 3);
}

public class MyThread
{
    public Thread Thrd;

    public MyThread(string name)
    {
        Thrd = new Thread(this.Run);
        Thrd.Name = name;
        Thrd.Start();
    }

    void Run()
    {
        for (int i = 0; i < 5; i++)
        {
            // Request a resource from the pool.
            MySharedResource.ObjSemaphore.WaitOne();

            // Get a random resource index.
            Random rnd = new Random();
            int index = rnd.Next(0, 3);

            // Operate on the resource.
            Console.WriteLine(Thrd.Name + " is operating on resource " + MySharedResource.resources[index]);
            Thread.Sleep(1000);

            // Release the resource to the pool.
            MySharedResource.ObjSemaphore.Release();
        }
    }
}

class Program
{
   static void Main()
   {
      Console.WriteLine("Main thread starting.");

      // Construct three threads.
      var mt1 = new MyThread("Child #1");
      var mt2 = new MyThread("Child #2");
      var mt3 = new MyThread("Child #3");

      mt1.Thrd.Join();
      mt2.Thrd.Join();
      mt3.Thrd.Join();

      Console.WriteLine("Main thread ending.");
   }
}

[参考]：

AutoResetEvent 和 ManualResetEvent 类

AutoResetEvent 和 ManualResetEvent 类是两种同步原语，用于通过信号来管理线程之间的同步。它们都继承自 EventWaitHandle 类，表示一个事件对象。

AutoResetEvent 类的特点是，当一个线程调用 WaitOne 方法等待信号时，如果信号量为终止状态 (true)，则该线程被允许继续执行，并且信号量自动重置为非终止状态 (false)。这样，每次只能有一个线程通过 WaitOne 方法。
ManualResetEvent 类的特点是，当一个线程调用 WaitOne 方法等待信号时，如果信号量为终止状态 (true)，则该线程被允许继续执行，并且信号量不会自动重置为非终止状态 (false)。这样，除非手动调用 Reset 方法将信号量设置为非终止状态 (false), 否则所有等待的线程都可以通过 WaitOne 方法。

一个简单的例子是，假设有三个工作线程 A、B、C 和一个主线程 M。主线程 M 需要在所有工作线程完成后才能结束。为了实现这个目的，可以使用一个 AutoResetEvent 对象和一个 ManualResetEvent 对象来同步主线程和工作线程之间的通知，如下所示：

using System;
using System.Threading;

public class MyWorkerThread
{
    public Thread Thrd;
    public AutoResetEvent ObjAuto;
    public ManualResetEvent ObjManual;

    public MyWorkerThread(string name, AutoResetEvent are, ManualResetEvent mre)
    {
        Thrd = new Thread(this.Run);
        Thrd.Name = name;
        ObjAuto = are;
        ObjManual = mre;
        Thrd.Start();
    }

    void Run()
    {
        Console.WriteLine("Inside thread " + Thrd.Name);

        for (int i = 0; i < 5; i++)
        {
            Console.WriteLine(Thrd.Name + " is doing some work.");
            Thread.Sleep(1000);
        }

        Console.WriteLine(Thrd.Name + " is signaling the main thread.");

        // Signal the main thread that this thread is done.
        ObjAuto.Set();

        // Wait for a signal from the main thread to continue.
        ObjManual.WaitOne();

        Console.WriteLine("Resuming thread " + Thrd.Name);

        for (int i = 0; i < 5; i++)
        {
            Console.WriteLine(Thrd.Name + " is doing some more work.");
            Thread.Sleep(1000);
        }

         Console.WriteLine(Thrd.Name + " is finished.");

         // Signal the main thread that this thread is done.
         ObjAuto.Set();
    }
}

class Program
{
   static void Main()
   {
      Console.WriteLine("Main thread starting.");

      // Construct an AutoResetEvent object and a ManualResetEvent object.
      var autoEvt = new AutoResetEvent(false);
      var manualEvt = new ManualResetEvent(false);

      // Construct three worker threads.
      var mt1 = new MyWorkerThread("Child #1", autoEvt, manualEvt);
      var mt2 = new MyWorkerThread("Child #2", autoEvt, manualEvt);
      var mt3 = new MyWorkerThread("Child #3", autoEvt, manualEvt);

      // Wait for all worker threads to finish their first phase of work.
      for (int i = 0; i < 3; i++)
      {
          Console.WriteLine("Main thread waiting for a signal.");
          autoEvt.WaitOne();
          Console.WriteLine("Main thread received a signal.");
      }

      // Signal all worker threads to resume their second phase of work.
      Console.WriteLine("Main thread signaling all worker threads to continue.");
      manualEvt.Set();

       // Wait for all worker threads to finish their second phase of work.
       for (int i = 0; i < 3; i++)
       {
           Console.WriteLine("Main thread waiting for another signal.");
           autoEvt.WaitOne();
           Console.WriteLine("Main thread received another signal.");
       }

       //

[参考]：

Use Pypy to make Python run faster

David Dong — Thu, 03 Jun 2021 07:04:44 +0800

“If you want your code to run faster, you should use Pypy.” - Guido van Rossum, the father of Python.
To understand why Pypy is faster, we should know what makes Python slower.

What does make Python slower?

Python is a dynamic programming language, as we all know, unlike the static programming language (C/C++/Rust…) - which are usually compiled ahead of time (AOT compilation), the dynamic programming language uses the interpreter to translate the source code line by line and runs it when executing. Therefore, The performance of the program interpreter determines the execution speed of Python.

For static programming language, the compiler will convert the source code into machine code readable by a specific computer architecture. In other words, when executing the program, the execution is not the source code, but the machine code.
Because of the lack of translation process, the execution of the static programming language is usually faster than a dynamic programming language.

Here are some common Python interpreters:

Interpreter	Description
CPython	interpreter implemented by C language
Jython	interpreter implemented by Java language
IronPython	interpreter implemented by .Net
Pypy	interpreter implemented by `RPython` - a subset of Python

What is Pypy?

We can think Pypy is a Python interpreter implemented in Python. Pypy is a very compatible Python interpreter, which is an alternative to CPython 2.7, 3.6, and the upcoming 3.7. It can significantly increase the speed when running an application with it.

Historically, Pypy has been used to mean two things. The first is the RPython translation toolchain for generating interpreters for dynamic programming languages. And the second is one particular implementation of Python produced with it. Because RPython uses the same syntax as Python, this generated version became known as Python interpreter written in Python. It is designed to be flexible and easy to experiment with.

Above is the introduction of the Pypy in the Pypy Documents website, this website offers a lot of technical details and introduction of the Pypy.

Why is Pypy faster?

The secret that the Pypy runs fast is that it uses JIT (Just-In-Time) compilation. JIT compilation is a combination of interpretation and pre-compilation. It can improve performance by using pre-compiling and improve the flexibility and cross-platform availability of interpretative language. That’s why Pypy can make Python perform so fast.

Here are the steps JIT compilation takes to provide faster performance:

Identify the most common components in code, such as functions in loops.
The runtime converts these components into machine code.
Optimize the generated machine code.
Replace the previous implementation with an optimized machine code version.

However, JIT isn’t all-powerful for everything. There are some limitations when using JIT.

It doesn’t work with C extensions, Pypy is best for pure Python applications. Whenever using the C extension module, it runs much slower than in CPython. The reason is that Pypy cannot optimize C extension modules because they are not fully supported. In this case, the Pypy team recommends removing the CPython extension and replacing it with a pure Python version. If not, CPython must be used.
It only works for long-running programs.
It does not perform precompilation. Pypy is not a fully compiled Python implementation. It compiles Python code, but it’s not a compiler for Python code. Python cannot be compiled into a separate binary and reused because of some inherent features of Python.

Use Pypy

The Pypy needs to be installed before using it.

The http://Pypy.org/ offered the installation ZIPs for running on the different platforms. It can be downloaded and installed.

To install Pypy under Ubuntu, another method is using the command line.

sudo add-apt-repository ppa:Pypy/ppa
sudo apt-get update
sudo apt-get install Pypy Pypy-dev

If the installation is successful, open the terminal and enter Pypy, there will be the following similar prompts.

Performance

To check the performance, I did an experiment, I wrote the below code and ran it with Pypy and CPython.

import time
t = time.time()
for i in range(10**8):
    continue
print (time.time() - t)

when using Pypy to run the code.

david@david-VirtualBox:~/Study/python$ Pypy test.py
0.148446083069

The result of using CPython to run the code.

david@david-VirtualBox:~/Study/python$ python3 test.py
19.179446935653687

The Pypy really made the python run faster!

Fixing the issue: Communication error with Jack server (35) when building AOSP

David Dong — Sun, 09 May 2021 07:14:11 +0800

I met an issue Communication error with Jack server (35) when built the AOSP yesterday. I can build the AOSP successfully before this issue popped up. I spent hours figuring out why and solved the problem.

Issue

My build environment of the AOSP is

Ubuntu 16.04 LTS server;
Android AOSP 8.0;
OpenJDK 8;

It works well until the below issue occurred yesterday. I remembered that I didn’t do anything unusual before that.

[  0% 1/20573] Ensuring Jack server is installed and started
FAILED: setup-jack-server 
/bin/bash -c "(prebuilts/sdk/tools/jack-admin install-server prebuilts/sdk/tools/jack-launcher.jar prebuilts/sdk/tools/jack-server-4.11.ALPHA.jar  2>&1 || (exit 0) ) && (JACK_SERVER_VM_ARGUMENTS=\"-Dfile.encoding=UTF-8 -XX:+TieredCompilation\" prebuilts/sdk/tools/jack-admin start-server 2>&1 || exit 0 ) && (prebuilts/sdk/tools/jack-admin update server prebuilts/sdk/tools/jack-server-4.11.ALPHA.jar 4.11.ALPHA 2>&1 || exit 0 ) && (prebuilts/sdk/tools/jack-admin update jack prebuilts/sdk/tools/jacks/jack-4.31.CANDIDATE.jar 4.31.CANDIDATE || exit 47 )"
Jack server already installed in "/home/david/.jack-server"
Communication error with Jack server (35), try 'jack-diagnose' or see Jack server log
SSL error when connecting to the Jack server. Try 'jack-diagnose'
SSL error when connecting to the Jack server. Try 'jack-diagnose'
ninja: build stopped: subcommand failed.
06:38:55 ninja failed with: exit status 1
build/core/main.mk:21: recipe for target 'run_soong_ui' failed
make: *** [run_soong_ui] Error 1
make: Leaving directory '/home/david/code/androidSrc'

#### make failed to build some targets (21 seconds) ####

Analysis

First, I tried to debug it by following the information in the logs. I ran the jack-diagnose command and it returned the below message.

error: process ID list syntax error

Usage:
 ps [options]

 Try 'ps --help <simple|list|output|threads|misc|all>'
  or 'ps --help <s|l|o|t|m|a>'
 for additional help text.

For more details see ps(1).
Port 8077 is used by another process (pid=), please ensure to free the port or change port configuration in '/home/david/.jack-settings' and '/home/david/.jack-server/config.properties'
error: process ID list syntax error

Usage:
 ps [options]

 Try 'ps --help <simple|list|output|threads|misc|all>'
  or 'ps --help <s|l|o|t|m|a>'
 for additional help text.

For more details see ps(1).
Port 8076 is used by another process (pid=), please ensure to free the port or change port configuration in '/home/david/.jack-settings' and '/home/david/.jack-server/config.properties

The log indicated the port has been used by another process, so I changed the port to another one by modifying the port number in the ~/.jack-settings and ~/.jack-server/config.properties.

However, it didn’t work. The issue was still existing.

Then I tried to restart the jack server by commands:

./prebuilts/sdk/tools/jack-admin kill-server
./prebuilts/sdk/tools/jack-admin start-server

The issue was still there!

There are some articles on the internet talking about this error, they pointed that the issue was caused by the multiple users doing the build process simultaneously, which is not supported by jack and will bring the issue. The solution is to change the port number to another one to keep jack working. But I don't think it is my case. And the result confirmed my idea.

OK! I kept investigating to get more information.

I checked the log in the file - ~/.jack-server/log/xxxx-0-0.log. It has nothing about the error.

I ran command ./prebuilts/sdk/tools/jack-admin dump-report and this time I got some useful information in the error report.

gnutls_handshake() failed: The TLS connection was non-properly terminated.

Looks like it is related to the TLS connection, so I ran more commands to check.

curl https://127.0.0.1:8076/jack
curl: (35) gnutls_handshake() failed: The TLS connection was non-properly terminated.

It verified that the issue is because of the TLS connection failure.

Fixing

Follow this clue, I searched Google to check if anyone has met the same issue with me. I finally found the post in stackoverflow that discussed this issue and provided the solution.

The reason for this issue is that I upgraded my OpenJDK to 8u292, but OpenJDK disabled the support for TLS 1.0 and 1.1, starting in 8u291. They think TLS 1.0/1.1 are old, insecure, and deprecated. Unfortunately, I was using TLS1.1 on my build system, therefore, the connection was broken for this reason.

I have two ways to solve this problem.

I can turn on the support of TLS1.0/1.1 manually by removing the TLSv1, TLSv1.1 declaring in jdk.tls.disabledAlgorithms configuration in file /etc/java-8-openjdk/security/java.security.
Switch to use TLS1.2 for the build system.

Solution 1 is relatively quick so I tried this way.

I enabled the support to TLS1.0/1.1, restart the jack server, and ran the build again.

Jack server already installed in "/home/david/.jack-server"
Server is already running
[100% 696/696] Install: out/target/product/bullhead/vendor/app/SensorTestTool/SensorTestTool.apk
make: Leaving directory '/home/david/code/androidSrc'

#### make completed successfully (03:51 (mm:ss)) ####

It worked this time!

If you’re in trouble with this issue like me, hope my experience and study can help you a little bit!

CI platform migration: from Travis CI to GitHub Actions

David Dong — Tue, 04 May 2021 20:14:11 +0800

I’ve used Travis CI to build and deploy my projects for some years. The performance of Travis CI can meet my requirements. However, recently I found Travis CI stopped working for me. ☹️

The motivation of migrating

The service of Travis CI provided convenience and helped me a lot with my development works during the past few years, I really appreciated it!

Until one day a couple of weeks ago, I received the notification from Travis CI, “My builds have been temporarily disabled for public repositories due to a negative credit balance”. I realized that the time of free use is over. I need to pay for the service now.

From the beginning, Travis CI was built to integrate with GitHub repositories and offer free open source CI. It was popular since then around the open-source developers. I think it is not only because of the powerful functionality but the free model of service. I knew Travis CI was purchased by Idera in Jan 2019. There should be some change in the pricing strategy. But when the new pricing strategy comes, I still feel a little bit frustrated.

A Free Plan, assigned automatically to every new sign up, is a Usage based plan with an unlimited amount of users which comes with a trial pool of credits to be used. Once these credits are used they are not replenished.

The new pricing plan offers 10000 credits for users to use freely and the credits will not be replenished once they are exhausted. At least 10 credits are paid for each build, which means the users only have a limited right of using the service now. I received the notice requesting payment because I have run out of my credits. I have to pay to continue using the service.

I checked the price of renewal, to be honest, it isn’t an attractive price. I think at least in China it isn’t.

To me, the reason for using Travis CI was because of the free model, and considering most of my projects on GitHub are non-commercial projects, I decided to abandon Travis CI and migrate my projects to another CI platform.

How about GitHub Actions?

GitHub published the CI system GitHub Actions in 2018, which can be thought of as a good substitute for Travis CI especially for the projects hosted on GitHub.

There are plenty of articles in the GitHub Docs to teach how to write your own GitHub Actions.

I have spent some hours reading through the below articles:

Travis CI and GitHub Actions are similar in terms of usage. They are all based on the YAML file format to write configuration files. The difference is GitHub Actions introduced the concept of the job in the workflows. Each job is separated into some steps, the step uses different actions to complete the function. So the users can set different actions according to different scenarios to reduce the complexity of development and speed up writing the configuration file.

This will provide great flexibility, I think that’s the most special thing about GitHub Actions.

Based on these features, many actions are developed by third-party developers for some common applications, such as grabbing code, running tests, logging on the remote server, publishing to third-party services, and so on. You can find them in the GitHub Marketplace.

If you want to start your CI, you don’t have to write complex actions by yourself. The quickest way is searching the GitHub Marketplace to find the actions that can fulfill the function you want and written by other contributors. You can construct the whole continuous integration process by combining actions.

Migrating to GitHub Actions

I decided to start the migration from my blog project. As I said above, both Travis CI and GitHub Actions use the YAML file format to write the configuration files, so the syntax is similar. What I do was rewriting the control flow with the GitHub Actions’ syntax and replaced the environment variables used in Travis CI.

The first thing was creating a .github/workflow folder in my blog’s repository then adding a YAML file.

It can be done by project repository's Action tab -> new workflow -> setup workflow with a template.

My blog’s CI process can be set for the below parts (see .travis.yml).

Setup build environment in the virtual machine.
Access permission authentication.
Jekyll Build
Deploy to GitHub Pages
Clean up

I don’t want to re-write the above procedure in GitHub Actions, I searched the GitHub Marketplace and found some actions that can help me complete the above works.

Because I need to do some extra works before deploying the static website files to GitHub Pages after they are generated by Jekyll. Therefore, what I want is just a build action, instead of the build & deploy integrated action.

I finally picked this action build-jekyll-toolbox to help me build the website.

After the build, I ported some of the deploy codes written in Travis CI to GitHub Actions. Since both of them use Linux OS on the virtual machine, the shell commands are identical, I finished it quickly. The change was to replace the GH_TOKEN environment variable with the DEPLOY_TOKEN in GitHub Actions (They are the same token just with the different name).

Below is the last code in my ci.yml file:

name: CI

# Controls when the action will run. 
on:
  # Triggers the workflow on push or pull request events but only for the master branch
  push:
    branches: [ dev ]
  pull_request:
    branches: [ dev ]

  # Allows you to run this workflow manually from the Actions tab
  workflow_dispatch:

# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
  jekyll:
    name: Build and deploy Jekyll site
    runs-on: ubuntu-latest

    steps:
    - name: Checkout
      uses: actions/checkout@v2

    - name: Build
      uses: ealenn/jekyll-build-action@v1
      
    - name: Deploy
      run: |
        git config user.name "gangdong"
        git config user.email "dqdongg@hotmail.com"
        chmod 777 .
        cp -r ./_site  /tmp/
        cp ./_site/blog/index.html /tmp/_site/
        git checkout -b deploy
        sudo rm -rf *
        cp -r /tmp/_site/* ./
        ls -la
        git add -A
        git commit -m "github_ci update gh-pages dev -> 2f3f1699 add Github Actions for build and deploy"
        git push --force "https://${{ secrets.DEPLOY_TOKEN }}@github.com/gangdong/gangdong.github.io.git" deploy:gh-pages

My feeling

With the above works, GitHub Actions started to build and deploy my blog website for me. After the trial, my immediate reflection is that GitHub Actions is fast build and simple to use.

You can see from the above source code, I used 2 actions checkout@v2 and jekyll-build-action@v1 to fetch the code and build them on the virtual machine. It does simplify the writing of configuration file compare to writing them in Travis CI.

As the above screenshot showed, each step of the process is clear and the whole construction costs ~ 1 minute to complete. It is a great improvement than running in Travis CI, which took 3 ~ 4 minutes for the whole process. Although I haven’t know what caused the promotion internally yet, It did improve a lot.

Badges

Like Travis CI, GitHub Actions supports status badges, which let you indicate whether a build is passing or failing.

For more information, see

Adding a workflow status badge to your repository.

I think GitHub Actions is an alternative option if you are looking for another CI to replace Travis CI.

Hope my practice was useful for you.

Speeding up my blog loadtime

David Dong — Sat, 10 Apr 2021 22:14:11 +0800

This is the story of how I speed up my blog website load time…

I’ve been adding some features to my blog which powered the website a lot. It is great! But, lately, the load time of website started to bother me. I was not happy with the website load time.

Current State

Today, I decided to go and figure out why. The first step was to gather data on why it was so slow. I will use Google Chrome Browser for the test and use DevTools to see what’s taking too much time to load. In order not to make this too extensive, I’ll keep only the top factors and the final time.

Let’s see how much time it takes right now:

approximate 25 seconds for the first-time connection. Let’s try to improve that.

As we can see, the top items that delay the response time are:

Factors	Duration	Percentage
Time for loading images	~19.7s	75%
Addthis_widgets.js	~19.54s	75%
Google Fonts	~3s	12%
Other JS/CSS files	~3.2s	13%

I will go to improve through them one by one.

Image Loading Time

I tried to adopt two methods to fast the image loading time.

Resize images to improve page load time:

The current solution is compressing the image size with the tool imagine, which can significantly compress the size of the image (up to more than 70%).

The future plan is to serve images in next-gen formats. Image formats like JPEG 2000, JPEG XR, and WebP often provide better compression than PNG or JPEG, which means faster downloads and less data consumption. The only concern is browser compatibility, not all of the browsers can support these formats so far.
If you want to know detailed information about the browser compatibility,
please view the below website:
- WebP
- JPEG 2000
- JPEG XR
Use image CDNs to optimize images:

Image content delivery networks (CDNs) are excellent at optimizing images.

The below contents replies to the question of What’s an image CDN?

Image CDNs specialize in the transformation, optimization, and delivery of images. You can also think of them as APIs for accessing and manipulating the images used on your site.

I chose a Third-party image CDN jsdelivr to do this work for me. It is free for open-source projects and has very good support to Github as they introduced in their official statement.

jsDelivr is a free CDN for open source files. We are tightly integrated with Github and npm allowing us to automatically provide a reliable CDN service to almost every open source project out there.

Especially In China:

jsDelivr has partnered up with multiple Chinese companies to provide fast and reliable file delivery in China mainland and the whole Asian continent. We have servers inside China that improve the delivery speeds and latency significantly. We also have a valid ICP license issued by the Chinese government that protects us from bans and slow downloads.
jsDelivr works perfectly inside China!

The usage of jsdelivr CDN is simple, I inserted the URL of jsdelivr service in the front of my local assets/images URL in the _config.yml by following Usage.

For more details of jsdelivr, refer to jsdelivr @Github.

OK, Let’s see the result with the above optimization.

Total ~ 160ms with two images loading! A good improvement I think! You may also find the image source has changed to jsdelivr CDN instead of my local Github Repos.

I am happy with the result and it is acceptable.

Addthis_widgets.js

I use Addthis widgets for the social media sharing of my posts. It is handy in functionality, I don’t want to delete it!

I decided to lazy load it at first, make sure a load of Addthis_widgets resource would not block the rendering of the index page.

The total load time reduced up to ~10s, but still, it takes too much time! Can I improve it even more?

After analysis, I found I introduced the Addthis_widgets.js in the default.html, which lead the Addthis_widgets.js was loaded at the first paint of the index page. However, the widgets weren’t used on the index page. If I move the loading of the JavaScript file to post.html where it displays the share button for the post, I will be able to eliminate the load time of Addthis_widgets.js completely! It will not affect the function.

I relocated the below code snippet into the bottom of post.html to load it.

<!-- Go to www.addthis.com/dashboard to customize your tools -->

<script async type="text/javascript" src="//s7.addthis.com/js/300/addthis_widget.js#pubid=ra-604f502a8198c9c9&domready=1"></script>

Readers will not use the share button until they finish the reading of the article. I think it has a negligible negative effect on user experience.

The test result showed Addthis_widgets.js is gone in the load phase, and the time went to 2.56s now!

That improved a lot. But I wanted more.

Google Fonts

Google Fonts are beautiful! I was using four fonts in my blog. I introduced them in the head.html, the load of fonts slows the page speed more or less.

However, after some works, I haven’t been able to find a good way to accelerate the acquisition of the fonts. I will keep my eyes on it!

Ohter JS/CSS

I reviewed all of the JS/CSS that block the page rendering.

I removed the unnecessary ones.
For the remained, I also use the jsdelivr CDN to speed up the acquiring.
Delivering critical JS/CSS inline and deferring all non-critical JS/styles.

Wow! After all of the above works, the load time is 1.84s finally! I can live with that.

The Last Test

I used some other tools to verify the final performance.

dotcom-monitor

I tested the load time from locations scattered on different continents worldwide. The test browser I chose Chrome.

The result is

The average time is about ~3s, exclude Buenos Aries, which is much slower (~6.7s).

I am more interested in the speed of the city I live in. The result is 2.6s.
Pingdom Website Speed Test
- Asia-Tokyo
- Europe-Frankfurt
- North America - USA-Washtington D.C

The Future Plan

Well, my further improvement plan is:

Try to use the new image format WebP/JPEG 2000. I will get extra 0.2s savings estimated with that.
Find equivalent of Google Font or load them locally.
Use CloudFlare CDN to speed up my page. Cloudflare has several neat features. It can minify files, improve image sizes, bundle js files, and so forth. By taking advantage of CloudFlare’s global network, I can utilize its CDN service to improve my site’s performance and security.
If considering the China local visiting I will perhaps change the NS to dnspod…

Reference

I would recommend some articles which are helpful for website load time optimization. I’ve read and referred them during the optimization of my blog website.

C# 线程

David Dong — Tue, 09 Mar 2021 21:55:22 +0800

在.NET 应用程序中，都是以 Main () 方法作为入口的，当调用此方法时系统就会自动创建一个主线程。

线程主要是由 CPU 寄存器、调用栈和线程本地存储器（Thread Local Storage，TLS）组成的。CPU 寄存器主要记录当前所执行线程的状态，调用栈主要用于维护线程所调用到的内存与数据，TLS 主要用于存放线程的状态信息。多线程的优点：可以同时完成多个任务；可以使程序的响应速度更快；可以让占用大量处理时间的任务或当前没有进行处理的任务定期将处理时间让给别的任务；可以随时停止任务；

可以设置每个任务的优先级以优化程序性能。那么可能有人会问：为什么可以多线程执行呢？总结起来有下面两方面的原因：

CPU 运行速度太快，硬件处理速度跟不上，所以操作系统进行分时间片管理。这样，从宏观角度来说是多线程并发的，因为 CPU 速度太快，察觉不到，看起来是同一时刻执行了不同的操作。但是从微观角度来讲，同一时刻只能有一个线程在处理。
目前电脑都是多核多 CPU 的，一个 CPU 在同一时刻只能运行一个线程，但是多个 CPU 在同一时刻就可以运行多个线程。

然而，多线程虽然有很多优点，但是也必须认识到多线程可能存在影响系统性能的不利方面，才能正确使用线程。不利方面主要有如下几点：

线程也是程序，所以线程需要占用内存，线程越多，占用内存也越多。
多线程需要协调和管理，所以需要占用 CPU 时间以便跟踪线程。
线程之间对共享资源的访问会相互影响，必须解决争用共享资源的问题。
线程太多会导致控制太复杂，最终可能造成很多程序缺陷。

C# 中创建线程

在 C# 中，线程是使用 Thread 类处理的，该类在 System.Threading 命名空间中。使用 Thread 类创建线程时，只需要提供线程入口，线程入口告诉程序让这个线程做什么。通过实例化一个 Thread 类的对象就可以创建一个线程。创建线程示例：

using System;  
using System.Threading;  

namespace MultithreadingApplication  
{  
    class ThreadCreationProgram  
    {  
        //编写线程要执行的方法
        public static void CallToChildThread()  
        {  
            Console.WriteLine("Child thread starts");  
        }  
         
        static void Main(string[] args)  
        {  
            //实例化一个线程对象，并传入一个指向线程所要执行方法的委托
            ThreadStart childref = new ThreadStart(CallToChildThread);  
            Console.WriteLine("In Main: Creating the Child thread");  
            Thread childThread = new Thread(childref); 
            //start线程 
            childThread.Start();  
            Console.ReadKey();  
        }  
    }  
}

上面代码实例化了一个 Thread 对象，并指明将要调用的方法 method ()，然后启动线程。 ThreadStart 是一个无参的、返回值为 void 的委托。

委托定义如下：

public delegate void ThreadStart()

通过 ThreadStart 委托创建并运行一个线程，ThreadStart 委托中作为参数的方法不需要参数，并且没有返回值。

创建多线程的步骤：
1、编写线程所要执行的方法； 2、实例化 Thread 类，并传入一个指向线程所要执行方法的委托。（这时线程已经产生，但还没有运行）;
3、调用 Thread 实例的 Start 方法，标记该线程可以被 CPU 执行了，但具体执行时间由 CPU 决定；

线程的同步

所谓同步：是指在某一时刻只有一个线程可以访问被保护的变量。如果不能确保对变量的访问是同步的，就会产生错误。 C# 为同步访问变量提供了一个非常简单的方式，即使用 C# 语言的关键字 Lock ，它可以把一段代码定义为互斥段，互斥段在一个时刻内只允许一个线程进入执行，而其他线程必须等待。在 C# 中，关键字 Lock 定义如下：

class Program
{
    static void Main(string[] args)
    {
        BookShop book = new BookShop();
        /创建两个线程同时访问saLe方法
        Thread t1 = new Thread(new Threadstart(book.Sale));
        Thread t2 = new Thread(new Threadstart(book.Sale));
        //启动线程
        t1.start();
        t2.Start();
        Console.ReadKey();
    }
}

class BookShop
{
    //剩余图书数量
    public int num = 1;
    public void Sale()
    {
        //使用Lock关键字解决线程同步问题
        lock (this)
        {
            int tmp = num;
            if(tmp>o)//判断是否有书，如果有就可以卖
            {
                Thread.sleep(1000);
                num -= 1;
                Console.WriteLine("售出一本图书，还剩余{e}本"，num);
            }
            else
            {
                Console.WriteLine("没有了")；
            }
        }
    }
}

C# 跨线程访问

假设我在一个新的子线程 mythread 上访问主线程的窗体控件 textBox, 会报错误 “线程间访问无效”，原因是 textBox 是由主线程创建的，mythread 线程是另外创建的一个线程，在.NET 上执行的是托管代码，C# 强制要求这些代码必须是线程安全的，即不允许跨线程访问 Windows 窗体的控件。

解决方案：使用回调函数。C# 的回调机制，实质上是委托的一种应用。

namespace MultiThreadDemo
{
    public partial class Form1 : Form;
    public Form1()
    {
        InitializeComponent();
    }
    //定义回调
    private delegate void setTextValueCallBack(int value);
    //声明回调
    private setTextValueCallBack setCallBack;
    private void btn_Test_click(object sender,EventArgs e)
    {
        //实例化回调
        setCallBack = new setTextValueCallBack(SetValue);
        //创建一个线程去执行这个方法：创建的线程默认是前台线程
        Thread thread = new Thread(new ThreadStart(Test));
        //Start,方法标记这个线程就绪了，可以随时被执行，具体什么时候执行这个线程，日
        //将线程设置为后台线程
        thread.IsBackground = true;
        thread.Start();
        private void Test()
        {
            for(int i=0；i<10000;i++)
            {
                //使用回调
                textBox1.Invoke(setCallBack,i);
            }
        }
    }
        ///<summary>
        ///定义回调使用的方法
        ///</summary>
        //<param name="vaLue"></param>
        private void Setvalue(int value)
        {
            this.textBox1.Text = value.ToString();
        }
    }
}

使用回调的步骤如下：

定义、声明回调方法。

//定义回调
private delegate void DoSomeCallBack(Type para);
//声明回调
DoSomeCallBack doSomaCallBack;

初始化回调方法，所谓“初始化回调方法”实际上就是实例化刚刚定义了的委托，这里作为参数的 DoSomeMethod 称为“回调方法”，它封装了对另一个线程中目标对象（窗体控件或其他类）的操作代码。
```
doSomeCallBack=new DoSomeCallBack(DoSomeMethod);
```

触发对象动作.

object Control.Invoke(Delegate method,params object[] args);

前台线程和后台线程

前台线程：只有所有的前台线程都结束，应用程序才能结束。默认情况下创建的线程都是前台线程。后台线程：只要所有的前台线程结束，后台线程自动结束。通过 Thread.IsBackground 设置后台线程。必须在调用 Start 方法之前设置线程的类型，否则一旦线程运行，将无法改变其类型。后台线程一般用于处理不重要的事情，应用程序结束时，后台线程是否执行完成对整个应用程序没有影响。如果要执行的事情很重要，需要将线程设置为前台线程。

线程常用的方法

Thread 中包括了多个方法来控制线程的创建、挂起、停止、销毁，以后来的例子中会经常使用。

方法	作用
Abort ()	终止本线程
Interrupt ()	中断处于 WaitSleepJoin 线程状态的线程
Join ()	阻塞调用线程，直到某个线程终止时为止
Resume ()	继续运行已挂起的线程
Start ()	执行本线程
Suspend ()	挂起当前线程，如果当前线程已属于挂起状态则此不起作用
Sleep ()	把正在运行的线程挂起一段时间

线程的常用属性值

属性值	说明
CurrentContext	获取当前线程的上下文
CurrentThread	获取当前正在运行的线程
isActive	获取当前线程的执行状态
isBackground	获取当前线程是否为后台线程
name	获取或者设置线程的名称
Priority	获取或者设置线程的优先级
ThreadState	获取线程的状态
ManagedThreadId	获取当前托管线程的唯一标识符

ManagedThreadId 是确认线程的唯一标识符，程序在大部分情况下都是通过 Thread.ManagedThreadId 来辨别线程的。开发人员可以通过程序设置线程的名称，但这只是一个辅助功能。

线程的优先级别

当线程之间争夺 CPU 时间时，CPU 按照线程的优先级给予服务。高优先级的线程可以完全阻止低优先级的线程执行。.NET 为线程设置了 Priority 属性来定义线程执行的优先级别，

里面包含 5 个选项，其中 Normal 是默认值。

Lowest: 可以将 Thread 安排在具有任何其他优先级的线程之后。
BelowNormal: 可以将 Thread 安排在具有 Normal 优先级的线程之后，在具有 Lowest 优先级的线程之前。
Normal: 默认选择。可以将 Thread 安排在具有 AboveNormal 优先级的线程之后，在具有 BelowNormal 优先级的线程之前。
AboveNormal: 可以将 Thread 安排在具有 Highest 优先级的线程之后，在具有 Normal 优先级的线程之前。
Highest: 可以将 Thread 安排在具有任何其他优先级的线程之前。

C# 中这条语句什么意思？

new Action(() => { this.comboBox1.SelectedItem.ToString(); })

这条语句是创建一个新的 Action 委托，它表示一个没有参数和返回值的方法。这个方法的内容是获取 comboBox1 控件中选中的项，并将其转换为字符串。这个委托可以用于 Invoke 或 BeginInvoke 方法，以便在其他线程上执行 UI 操作。例如

Invoke(new Action(() => { this.comboBox1.SelectedItem.ToString(); }))

new Action 的用法是什么？如果我想表示一个 string 类型的返回值，没有参数的方法该如何实现？

new Action 的用法是创建一个委托类型，它可以表示一个没有返回值的方法。如果你想表示一个 string 类型的返回值，没有参数的方法，你可以使用 Func<string> 委托。例如：

// 创建一个Action委托，它表示一个打印Hello World的方法
Action action = new Action(() => {Console.WriteLine("Hello World");});
// 调用这个委托
action();

// 创建一个Func<string>委托，它表示一个返回当前日期字符串的方法
Func<string> func = new Func<string>(() => {return DateTime.Now.ToString();});
// 调用这个委托，并打印返回值
Console.WriteLine(func());

A quick way to fix the Gitalk Error: Validation Failed

David Dong — Mon, 08 Mar 2021 10:27:13 +0800

I’ve met an error Validation Failed (422) when using Gitalk in my blog. This article records how I fixed this error. If you are in trouble with this issue and you are looking for a solution, reading this article may help you.

I posted a new article in my blog and found the Gitalk report on such an issue. My Gitalk has been working for a long time before this issue occurs. I’ve checked that the Gitalk for my other posts works normally and only the new post has the issue, which means the issue should be a standalone problem that is related to the post self.

By searching google - gitalk issue #102 I understood the problem is that the length of my new post’s URL is excessive long (> 50 characters).

For each post, Gitalk created a Github issue under my blog project’s repository to track the comment thread of the post. The issue’s id is used to generate the issue’s label which is used by Gitalk to identify the comment thread.

However, the problem here is that label length is restricted to maximum of 50 characters ( Not sure if it is a hidden rule or a BUG here 😕 ).

In my Gitalk configuration, the issue id is set to

id: { {page.url} },

So if the page.url is too long and is beyond the restriction, the issue’s label cannot be created and the Validation Failed error will be thrown out.

The root cause is clear now, let me fix it.

The straightforward way is to use a shorter URL to avoid this issue. But I don’t like this restriction that blocking me write a post. To convert the URL to a fixed-length string looks like a better solution. Why not try the hash function? The interesting thing is that I found there has already been the same solution online -> issue fix, which uses the MD5 method. This method not only unifies the URL length but also differentiates them.

It is good!

There is a workable MD5 library - JavaScript-MD5 on Github, I folk it, and the rest thing is quite simple.

I added the below code snippet for loading the JavaScript-MD5 library in my comments.html

<script type="text/javascript" src="https://cdn.jsdelivr.net/gh/gangdong/gangdong.github.io@dev/assets/js/md5.min.js"></script>

and reset the Gitalk id to

id: md5(location.pathname),

Committed the update.

I reopened the page and the error was gone, by checking the Gitalk issue’s label, it has been converted to MD5 code.

This means the solution is working now! 😊

Build TA images on different TEE

David Dong — Thu, 04 Feb 2021 12:21:47 +0800

This article will give an introduction to how to build TA images on different TrustZone.

1. About TEE

Nowadays, many security-related applications must run on the TEE. TEE (Trusted Execution Environment) can provide an absolute-safe environment for any user security requirement. Fingerprint application as one kind of biometric authentication, it is a trusted application and must run on the TEE. Here I will take the Fingerprint application as an example to introduce how to build the trusted application (TA) image.

Before start, I want to use the below diagram for presenting a short description on how TEE works on fingerprint applications.

The main control flow, handling the various use cases, is implemented in REE (Fingerprint HAL). Images from the sensor are captured on the TEE side and managed by fingerprint library, which also coordinates data flow towards the submodules implementing various algorithms for image processing and biometric processing. Enrolled fingerprint templates are managed in a RAM database in Fingerprint TA, and encrypted before passed to REE side for persistent storage. when authentication occurred, the matcher algorithm on the TEE side will work and give the matching result to REE. The communication channel - SPI transmission is physical in TEE and normally works by calling TEE API.

The software module working on the TEE side is normally built into a binary file, which runs on the TEE OS as an executable application (Trusted Application). Different TEE OS can support a different number of Trusted Application (TA).

There is multiple commercial TEE OS on the Android platform supported by third-party companies, some popular ones among them are QSEE, ISEE, Trusty. The following content will introduce how the fingerprint TA image is built out on these TEE OS.

2. QSEE 5

QSEE is one TEE OS supported by Qualcomm. In the android market, the system running with the Qualcomm platform uses the QSEE Trust zone. QSEE provides a set of SDK that helps the developer to develop the TEE application and generate the image file (TA image).

2.1 SDK

To build the TA image, we need to use QSEE SDK, which can be got from Qualcomm or ODMs. Here I use QSEE5 SDK and put it to below location in my unbuntu.

/home/david/devtools/tz_qsee5

2.2 Code

Next we need to put the source code of Fingerprint TA into place that the SDK can find it and make. In QSEE, normally there is specific location in the SDK file tree for storing the TA code. It is at path:

/home/david/devtools/tz_qsee5/ssg/securemsm/trustzone/qsapps/

2.3 Build

We execute the build command to make the TA code and generate the TA image.

cd /home/david/devtools/tz_qsee5/build/ms/
python build_all.py -b TZ.XF.5.0.1 CHIPSET=sdm845 --cbt="$(FPC_CONFIG_TZ_IMAGE_NAME)" $(build_flags)

Build process

2.4 Image

TA images are generated at path:

/home/david/devtools/tz_qsee5/build/ms/bin/PIL_IMAGES/SPLITBINS_WAXAANAA/

The TA images are

fpctzappfingerprint.b00
fpctzappfingerprint.b01
fpctzappfingerprint.b02
fpctzappfingerprint.b03
fpctzappfingerprint.b04
fpctzappfingerprint.b05
fpctzappfingerprint.b06
fpctzappfingerprint.b07
fpctzappfingerprint.mdt

3. ISEE

There is no property TEE OS under the MTK platform. It adopts the way of integrating the TEE environment of a third-party. The common TEE manufacturers ISEE.

3.1 SDK

For ISEE SDK, it can find more details on this link 「ISEE SDK」 I put the SDK into the below location:

/home/david/devtools/isee_sdk_270

3.2 Code

Put the TA source code to the below path:

/home/david/devtools/platforms/mt6797/vendor/fingerprints/

3.3 Build

Run the command

cd /home/david/devtools/isee_sdk_270
source setenv.sh 
cd /home/david/devtools/platforms/mt6797/vendor/fingerprints/fingerprint_ta/secure/platform/isee
 make

Build process

3.4 image

TA image is generated at the path:

/home/david/devtools/platforms/mt6797/vendor/fingerprints/fingerprint_ta/secure/platform/isee/obj/7778c03fc30c4dd0a319ea29643d4d4b.ta

It uses the UUID of the TA as the TA name, which is UUID.ta

4. Trusty

Trusty TEE is originated and supported by Google, which is integrated into the android as a secure Operating System (OS) that provides a Trusted Execution Environment (TEE). For more details about Trusty, please refer to 「Trusty TEE」

4.1 SDK

Find a location in your local device. For example, I put the trusty SDK here:

/home/david/devtools/trusty_sdk

4.2 Code

Copy the TA source code to the SDK folder.

/home/david/devtools/trusty_sdk/app/demo/

4.3 Build

Run the command

cd /home/david/devtools/trusty_sdk
make M="app/demo/fpctzapp:TA"

Build process

4.4 image

The TA image is generated at below location after compiling completed.

/home/david/devtools/trusty_sdk/build/user_tasks/app/demo/fpctzapp/fpctzapp.elf
/home/david/devtools/trusty_sdk/build/user_tasks/app/demo/fpctzapp/fpctzapp.elf/fpctzapp.syms.elf (for debug use)

It uses the TA name that is defined at the configuration as the image name, which is TA_Name.elf. Such as fpctzapp.elf. The TA_Name.syms.elf is the image file that containing the symbols table which can be used for debugging purposes.

Some useful knowledge for Fingerprint Application on Trusty TEE

David Dong — Wed, 02 Dec 2020 22:24:12 +0800

I have completed an Android platform fingerprint application on Trusty TEE recently for my work. This page will try to give some useful information, which is a summary of my work and might be helpful for somebody who wants to bring up a fingerprint application on Trusty TEE.

1. Trusty TEE

As one of the biometric authentication on the Android platform, fingerprint implementation must meet android security specifications. Android uses a separate secure Operating System (OS) to guarantee the security of biometric applications - we call it TEE (Trusted Execution Environment), Which runs on the same processor as the Android OS and is isolated from the rest of the system by both hardware and software. They run parallel to each other but secure OS has access to the full power of a device’s main processor and memory but is completely isolated.

There is multiple commercial TEE OS on the Android platform supported by third-party companies, Such as QSEE, ISEE, TBase, and so on. Trusty is one of them and unlike them, Trusty TEE is supported by Google. Trusty TEE is trying to provide the users a reliable and free open source alternative for their Trusted Execution Environment.

Google official documents provides more information about 「Trusty TEE」.

2. Memory restriction

2.1 Memory

The total memory that Trusty TEE can provide is 32M, suggests allocate 10M memory (heap + stack + ta image) for fingerprint to use. For example, using 6M heap and 3M stack.

2.2 Buffer

a. The communication between CA and TA is limited in size, and the overall size is limited to 128KB, including message header. Therefore, the buffer size TAC_SHARED_BUFFER_SIZE should be less than 128K.

#define TAC_SHARED_BUFFER_SIZE 1024 * 120 //should not be greater than 128k

b. Accordingly, the actual data size that can be used for effective transmission between CA and TA is limited to

1024 x 128 - sizeof(union ta_target_commands) - sizeof(int32_t)

/* ISEE may need extra room for their own header */
#if defined(ISEE)
#define MAX_CHUNK ((SECURE_BUFFER_MAX_SIZE) - (MAX_COMMAND_SIZE) - 64)
#elif defined(TOS)
#define MAX_CHUNK ((SECURE_BUFFER_MAX_SIZE) - (MAX_COMMAND_SIZE) - 4)
#else
#define MAX_CHUNK ((SECURE_BUFFER_MAX_SIZE) - (MAX_COMMAND_SIZE))
#endif

c. When the data to be transferred is greater than the maximum limit, consider transferring in batches.
d. Change the size of heap and stack in the manifest file.

trusty_app_manifest_t TRUSTY_APP_MANIFEST_ATTRS trusty_app_manifest =
   {
 
        /* UUID : {4304bef6-36e5-4d90-94b0-1ea4cd51d40b} */
       /* { 0x4304bef6, 0x36e5, 0x4d90,
        { 0x94, 0xb0, 0x1e, 0xa4, 0xcd, 0x51, 0xd4, 0x0b } }, */
      { 0x4304bef6, 0x36e5, 0x4d91,
        { 0x94, 0xb0, 0x1e, 0xa4, 0xcd, 0x51, 0xd4, 0x0c } },
 
     /* optional configuration options here */
    {
        /*apply 6M heap and 3M stack*/
       TRUSTY_APP_CONFIG_MIN_HEAP_SIZE(6*1024*1024),
       TRUSTY_APP_CONFIG_MIN_STACK_SIZE(3 * 1024 * 1024),
      },
    };

3. TEE Communication

Adopt the dynamic TA mechanism which will load TA and run TA’s main function when CA calls function connect(). When CA calls disconnect() the TA process exits. Therefore, in a life cycle, there is no need to connect or disconnect each IPC communication.
There are many IPC communications between CA and TA. Every time IPC communication, the buffer received and sent by CA needs to be reallocated. The same buffer should not be used by IPC multiple times. In our code, this method has already been implemented.
Trusty TEE provides 2 ports for communication, secure port and non-secure port.
Secure port - for other TA app access.
Non secure port - for CA access TA app.
For fingerprint, it needs to use non-secure port and if has payment requirement, needs to use secure port.
Should define the same port name between CA and TA, An example that we are using “com.android.trusty.fpctzapp”.
Should use unique UUID to differentiate from other fingerprint vendors.
About IPC: the Trusty APIs use

send_msg()
get_msg()
read_msg()
put_msg()

to send/retrieve messages between CA and TA, the calling sequence should be correct. One lesson learned in my software bring up is that the communication was failed after executed one-time successful communication. The communication was hang up after then and TA wasn’t able to get the message from CA. The failure was due to missing the put_msg() calling after executed read_msg().

static long handle_msg(tzapp_chan_ctx_t* ctx)
{
handle_t chan = ctx->chan;
 
/* get message info */
ipc_msg_info_t msg_inf;
uint8_t* msg_buf = NULL;
long rc = get_msg(chan, &msg_inf);
if (rc == ERR_NO_MSG)
return NO_ERROR; /* no new messages */
 
// fatal error
if (rc != NO_ERROR) {
LOGE("Trusty: failed (%ld) to get_msg for chan (%d), closing connection", rc, chan);
return rc;
}
 
//MessageDeleter md(chan, msg_inf.id);
 
// allocate msg_buf, with one extra byte for null-terminator
msg_buf = (uint8_t*) malloc(msg_inf.len + 1);
LOGD("Trusty: handle_msg msg_inf.len = %d", msg_inf.len);
if (!msg_buf) {
LOGE("Trusty: msg_buf failed to malloc");
return ERR_NO_MEMORY;
}
 
/* read msg content */
struct iovec iov = {msg_buf, msg_inf.len};
ipc_msg_t msg = {1, &iov, 0, NULL};
 
memset(msg_buf, 0, msg_inf.len + 1);
rc = read_msg(chan, msg_inf.id, 0, &msg);
 
int rc_ = put_msg(chan, msg_inf.id); // change to put_msg more early
 
// fatal error
if (rc < 0) {
LOGE("Trusty: failed to read msg (%ld)", rc);
goto out;
}
 
if (((unsigned long)rc) < msg_inf.len) {
LOGE("Trusty: invalid message of size (%ld)", rc);
rc = ERR_NOT_VALID;
goto out;
}
 
if (rc_ != NO_ERROR)
{
LOGE("Trusty: failed (%d) to put_msg for chan (%d)\n", rc, chan);
return rc_;
}
 
int32_t rsp;
void* addr = (void *)msg_buf;
rsp = ctx->dispatch(ctx, addr, msg_inf.len);
 
rc = send_response(chan, addr, msg_inf.len, &rsp);
out:
if (msg_buf) {
free(msg_buf);
}
return rc;
}

4. SPI

It is related to the hardware platform, on Spreadtrum SC9863, it doesn’t need to configure SPI and will only use ioctl() for transmission.

5. Others

5.1 How to build ?

5.1.1 Toolchain
It is recommended to use the arm-eabi-4.8 tool chain of Android code package:

export PATH=$PATH:<AOSP>/prebuilts/gcc/linux-x86/arm/arm-eabi-4.8/bin

5.1.2 Build
put the TA code fpctzapp into SDK app/demo/ folder.
run command

make M="app/demo/fpctzapp:TA"

5.1.3 Output Image
output two image files fpctzapp.elf and fpctzapp.syms.elf (which contains symbol table for debug purpose)

5.2 Tools

5.2.1 uuidgen
Output two image files fpctzapp.elf and fpctzapp.syms.elf (which contains symbol table for debug purpose)
5.2.2 addr2line
To find the line number of error occurrence from symbol table. In the

bsp/toolchain/prebuilts/gcc/linux-x86/arm/arm-eabi-4.8/bin/ folder

5.2.3 signta.py
Signature tool for signing the TA image. In the

vendor/sprd/proprietories-source/packimage_scripts/signimage/dynamicTA/"

python signta.py --uuid {UUID} --key “privatekey.pem” --in “TA image name without signed” --out “signed TA image name”.

command for signature.

python signta.py --uuid {UUID} --key “privatekey.pem” --in “TA image name without signed” --out “signed TA image name”

5.3 Logs

5.3.1 TA load successfully

 [ 68.183207] c4 246 trusty: ta_manager_wait_load:382: ta_manager_wait_load com.android.trusty.fpctzapp 
 [ 68.185949] c4 246 trusty: ta_manager_write_ta:485: ta_manager_write_ta: new ta! 
 [ 68.188528] c0 181 trusty: ta_manager_write_ta:573: ta_manager_write_ta, load com.android.trusty.fpctzapp accomplished!

5.3.2 Failure with TA wasn’t signed or signatue wasn’t match

 [ 30.866766] c1 trusty: ta_manager_write_ta:538: ta_manager_write_ta: new ta! 
 [ 30.999062] c0 trusty: ta_manager_handle_msg:760: ta_manager_handle_request failed -17!

5.3.3 TA APP wasn’t running properly, CA lost communication

 libtrusty: tipc_connect: can't connect to tipc service "com.android.trusty.fpctzapp" (err=107)

利用 Microsoft WDK 工具生成数字签名

David Dong — Thu, 23 Jul 2020 18:24:12 +0800

看过我这一篇文章「浅谈数字签名」的读者应该记得，在这篇文章的末尾遗留了一个问题 — 在windows平台下如何利用Microsoft提供的工具来生成数字签名。我在这篇文章中会对这个问题做一个说明，并引用一个例子来介绍签名的过程。

1. 相关工具

这里我先把和签名相关的工具罗列出来。

工具	说明
makecert.exe	使用这个工具来生成测试用的证书
cert2spc.exe	使用 cert2spc.exe 将公钥证书转换为软件发布者证书，即spc文件
pvk2pfx.exe	使用 pvk2pfx.exe 将公钥证书和私钥证书合并成一个PFX格式的证书文件
inf2cat.exe	驱动开发会用到，该工具确定驱动程序包的 INF 文件是否可以针对指定的 Windows 版本列表进行数字签名。如果可以，那么 Inf2Cat.exe 会生成适用于指定 Windows 版本的未签名的目录文件
signtool.exe	数字签名制作工具,制作数字签名和验证

2. 制作流程

使用 Inf2Cat.exe 将INF转成CAT 目录文件（针对驱动开发，非驱动程序可跳过这步）
使用 makecert.exe 制作自己的根证书
使用 cert2spc.exe 将公钥证书转换为软件发布者证书，即spc文件
使用 pvk2pfx.exe 将公钥证书和私钥证书合并成一个PFX格式的证书文件
使用 signtool.exe 签名
使用 signtool.exe 验证签名

3. 举例

下面我就以一个例子来演示一下整个过程。

3.1 生成目录文件（*.cat）

利用 Inf2Cat.exe 将INF 转换为 CAT 目录文件这部分内容，我在这一篇文章 Inf2Cat 工具使用中已经做过介绍。这里不在进行说明。

3.2 生成数字证书

makecert.exe 是一种证书创建工具，生成仅用于测试目的的 X.509 证书。此工具将密钥对与指定发行者的名称相关联，并创建一个 X.509 证书，该证书将用户指定的名称绑定到密钥对的公共部分。

makecert.exe 的存放路径

C:\Program Files (x86)\Windows Kits\10\bin\10.0.17763.0\x86

如果没有设置该路径的环境变量的话，使用时要先切换到该路径下。或者直接使用Visual Studio 的开发人员命令提示 Developer Command Prompt 执行命令。

关于 makecert.exe 的命令格式和参数，请见附录「makecert.exe 命令格式」。

执行下列命令创建一个创建自签署证书 RootDavid。

makecert -n "CN=RootDavid" -r -sv testsk.pvk testpk.cer

运行这个命令后，会弹出提示框，首先给 testsk.pvk 文件设置私钥保护口令。

然后，再次输入这个口令用私钥(testsk.pvk文件中）来给公钥（testpk.cer文件中）签名（自签名）。

输入正确的私钥口令后，控制台会返回 Succeeded. 并在当前目录下生成两个文件

其中

私钥证书 - testsk.pvk
公钥证书 - testpk.cer

如果你需要用这个根证书签发其他子证书的话，可以运行如下命令签发。

makecert -n "CN=SubCertDavid" -iv testsk.pvk -ic testpk.cer -sv subsk.pvk subpk.cer

3.3 将公钥证书格式转换成SPC（软件发布者证书）

使用 cert2spc.exe 这个工具将刚才生成的公钥证书（testpk.cer）转换为 SPC 文件。
cert2spc.exe 的存放路径

C:\Program Files (x86)\Windows Kits\10\bin\10.0.17763.0\x86

关于 cert2spc.exe 的用法，请见附录「cert2spc.exe 命令格式」。

运行如下命令

C:\Program Files (x86)\Windows Kits\10\bin\10.0.17763.0\x86\cert2spc testpk.cer testpk.spc

执行成功后在当前目录下生成

3.4 将公钥证书和私钥合并成一个PFX格式的证书文件

使用 pvk2pfx .exe 这个工具将公钥证书（testpk.spc）和私钥证书(testsk.pvk)合并成一个PFX格式的证书文件。
pvk2pfx.exe 的存放路径

C:\Program Files (x86)\Windows Kits\10\bin\10.0.17763.0\x86

关于 pvk2pfx.exe 的用法，请见附录「pvk2pfx.exe 命令格式」。

执行如下命令

C:\Program Files (x86)\Windows Kits\10\bin\10.0.17763.0\x86\pvk2pfx -pvk testsk.pvk -spc testpk.spc -pfx testpfx.pfx

合并时会要求输入私钥testsk.pvk的保护口令来合并.pvk和.spc文件。输入之前设定的口令，就可在当前目录下得到生成的pfx文件。

3.5 生成签名

使用signtool命令签名。

signtool.exe 的存放路径

C:\Program Files (x86)\Windows Kits\10\bin\10.0.17763.0\x86

关于signtool的用法，请见附录「signtool.exe 命令格式」。

执行下列命令。

signtool sign /v /f testpfx.pfx /tr http://timestamp.digicert.com MXT_ENC2Array_Converter.exe

如果成功，会返回

这时签名已经成功了。我们打开被签名的应用程序的属性标签，找到数字签名那一栏，可以看到签名的信息。

点开 详细信息-> 高级，可以看详细的签名信息。

X.509 格式证书信息。

证书颁发路径为根证书。

我们看到当前的证书是不授信的，需要将其添加导入证书库中。
Microsoft规定应使用证书存储区来安全地存储证书（证书存储区是系统中一个特殊区域，专门用来保存X.509数字证书）。

Windows 预设了以下存储区：

存储区	说明
AddressBook	其他用户的 X.509 证书存储区。
AuthRoot	第三方证书颁发机构 (CA) 的 X.509 证书存储区。
CertificateAuthority	中间证书颁发机构 (CA) 的 X.509 证书存储区。
Disallowed	吊销的证书的 X.509 证书存储区。
My	个人证书的 X.509 证书存储区。
Root	受信任的根证书颁发机构 (CA) 的 X.509 证书存储区。
TrustedPeople	直接受信任的人和资源的 X.509 证书存储区。
TrustedPublisher	直接受信任的发行者的 X.509 证书存储区。

下面我们就将证书添加到存储区，选择 安装证书，然后一路 Next。

直到 证书存储 步骤，选择 将所有的证书放入下列存储：点击 浏览，在弹出的对话框中选择 个人-> 确定。

为什么要选择 个人 呢？可以参见 signtool 的 sign 子命令的/s 选项的说明：「指定要在搜索证书时打开的存储区。如果未指定该选项，则打开My存储」这里的 My 就是 个人。然后就可以一路 下一步 到导入完成了。我们可以到计算机的管理控制台确认。在开始菜单中搜索并运行mmc。在mmc界面中，选择 文件-> 添加删除管理单元。在弹出的 添加删除管理单元 对话框中，在左边的 可用的管理单元 中选择 证书。

点击中间的 添加 按钮，在弹出的对话框中选择 我的用户帐户 或 计算机用户帐户，再点击 完成：

就将 证书 节点添加到 所选管理节点 中了：

点击 确定，回到管理控制台主界面中，在左边的树控件中展开 证书 -> 当前用户-> 个人，选择 证书 节点，就可以看见已经导入的RootDavid证书。

双击RootDavid，可以看见证书对话框里写着“您有一个与该证书对应的私钥”。对话框里还写着“此CA根目录证书不受信任。要启用信任，请将该证书安装到 受信任的根证书颁发机构 存储区”。后面会讲到这一点。

3.6 验证签名

接下来就是要验证 exe 的签名。

执行下列命令

signtool verify /pa MXT_ENC2Array_Converter.exe

此时返回了如下的结果，显示

这是因为目前还没有将 RootDavid 的公钥证书添加到之前提到的 受信任的根证书颁发机构。要做这一步的话，需要找到公钥证书 testpk.cer，然后双击该cer文件，在弹出的证书对话框中选择 安装证书。接下来和之前的导入证书操作一样，只是在 证书存储 这一步，需要选择将证书存储在 受信任的根证书颁发机构 存储区，而不是之前的 个人 存储区中。

此时会弹出一个警告提示，让你确认是否要安装此证书到授信存储区，点击确认后，成功安装。回到管理控制台主界面中，在左边的树控件中展开 证书 -> 当前用户 -> 受信任的根证书颁发机构，选择 证书 节点，就可以看见已经导入的 testpk.cer 公钥证书。
此时在进行验证，验证成功！:100:

此时再打开被签名的应用程序属性页，可以看到签名已经是授信的了。

4. 常见问题

执行 makecert.exe 时出现如下错误

   'makecert' is not recognized as an internal or external command,
   operable program or batch file.

原因：这是因为没有在 makecert.exe 的路径下执行该命令，解决方法时切换到工具所在的目录下执行。

   cd C:\Program Files (x86)\Windows Kits\10\bin\10.0.17763.0\x86

或者直接使用 Visual Studio 的开发人员命令提示 Developer Command Prompt 运行命令。

我在执行 makecert.exe 时失败，返回如下提示

   Error: File already exists for the subject ('testsk.pvk')
   Error: Can't create the key of the subject ('testsk.pvk')
   Failed

原因：这是因为没有在管理员权限下执行 makecert.exe 命令，解决方法是用管理员权限打开 CMD，运行命令。

我的需要签名的文件名里有空格，执行签名的操作后失败。

   signtool sign /v /f testpfx.pfx /tr http://timestamp.wosign.com/rfc3161 my file name.exe

原因：文件名不能包含空格，重命名后再次签名可以成功。

执行 signtool verify 时提示原因：这是因为公钥文件xx.cer没有被添加到受信任的根证书颁发机构。解决方法如上。

5. 附录

5.1 makecert.exe

makecert.exe 命令格式

MakeCert [/b DateStart] [/e DateEnd] [/len KeyLength] [/m nMonths] [/n "Name"] [/pe] [/r] [/sc SubjectCertFile] [/sk SubjectKey] [/sr SubjectCertStoreLocation] [/ss SubjectCertStoreName] [/sv SubjectKeyFile]OutputFile

5.1.1 基本选项

选项	说明
`-n` x.509name	指定主题的证书名称。在双引号中指定此名称，并加上前缀 CN=；例如，”CN=myName”。
`-pe`	将所生成的私钥标记为可导出。这样可将私钥包括在证书中。
`-sk` keyname	指定主题的密钥容器位置，该位置包含私钥。如果密钥容器不存在，系统将创建一个。
`-sr` location	指定主题的证书存储位置。Location 可以是 currentuser（默认值）或 localmachine。
`-ss` store	指定主题的证书存储名称，输出证书即存储在那里。
`-#` number	指定一个介于 1 和 2,147,483,647 之间的序列号。默认值是由 Makecert.exe 生成的唯一值。
`-$` authority	指定证书的签名权限，必须设置为 commercial（对于商业软件发行者使用的证书）或 individual（对于个人软件发行者使用的证书）。

5.1.2 扩展选项

选项	说明
`-a` algorithm	指定签名算法。必须是 `md5`（默认值）或 `sha1`。
`-b` mm/dd/yyyy	指定有效期的开始时间。默认为证书的创建日期。
`-e` mm/dd/yyyy	指定有效期的结束时间。默认为 `12/31/2039 11:59:59 GMT`。
`-m` number	以月为单位指定证书有效期的持续时间。
`-cy` certType	指定证书类型。有效值是 `end`（对于最终实体）和 `authority`（对于证书颁发机构）。
`-d` name	显示主题的名称
`-eku` oid[,oid]	将用逗号分隔的增强型密钥用法对象标识符 (OID) 列表插入到证书中。
`-h` number	指定此证书下面的树的最大高度。
`-ic` file	指定颁发者的证书文件。
`-ik` keyName	指定颁发者的密钥容器名称。
`-iky` keytype	指定颁发者的密钥类型，必须是 exchange、signature 或一个表示提供程序类型的整数。默认情况下，可传入 1 表示交换密钥，传入 2 表示签名密钥。
`-in` name	指定颁发者的证书公用名称。
`-ip` provider	指定颁发者的 CryptoAPI 提供程序名称。
`-ir` location	指定颁发者的证书存储位置。Location 可以是 currentuser（默认值）或 localmachine。
`-is` store	指定颁发者的证书存储名称。
`-iv` pvkFile	指定颁发者的 .pvk 私钥文件。
`-iy` pvkFile	指定颁发者的 CryptoAPI 提供程序类型。
`-l` link	到策略信息的链接（例如，一个 URL）。
`-r`	创建自签署证书。
`-sc` file	指定主题的证书文件。
`-sky` keytype	指定主题的密钥类型，必须是exchange、 signature或一个表示提供程序类型的整数。默认情况下，可传入 1 表示交换密钥，传入 2 表示签名密钥。
`-sp` provider	指定主题的 CryptoAPI 提供程序名称。
`-sv` pvkFile	指定主题的 .pvk 私钥文件。如果该文件不存在，系统将创建一个。
`-sy` type	指定主题的 CryptoAPI 提供程序类型。

5.2 cert2spc.exe

软件发行者证书测试工具 cert2spc.exe 从一个或多个X.509证书创建软件发行者证书（SPC）。cert2spc.exe 仅用于测试目的。商业目的的SPC可以从证书颁发机构（如VeriSign或Thawte）获取。

5.2.1 语法格式

cert2spc cert1.cer | crl1.crl [... certN.cer | crlN.crl] outputSPCfile.spc

5.2.2 参数

参数	说明
`certN.cer`	要包含在 SPC 文件中的 X.509 证书的名称。可以指定用空格分隔的多个名称。
`crlN.crl`	要包含在 SPC 文件中的证书吊销列表的名称。可以指定用空格分隔的多个名称。
`outputSPCfile.spc`	将包含 X.509 证书的 PKCS #7 对象的名称。

5.2.3 选项

选项	说明
/?	显示该工具的命令语法和选项。

5.3 pvk2pfx.exe

Pvk2Pfx.exe 是一个命令行工具，它将 .spc、.cer 和 .pvk 文件中包含的公钥和私钥信息复制到个人信息交换（.pfx）文件中。命令格式

pvk2pfx /pvk pvkfilename.pvk [/pi pvkpassword] /spc spcfilename.ext [/pfx pfxfilename.pfx [/po pfxpassword] [/f]]

5.3.1 参数

参数	说明
`/pvk` pvkfilename.pvk	指定一个.pvk 文件的名称。
`/spc` spcfilename.ext	指定的名称和扩展名软件发布者证书 (SPC)包含证书的文件。该文件可以是.spc 文件或 `.cer` 文件。
`/pfx` pfxfilename.pfx	指定.pfx 文件的名称。
`/pi` pvkpassword	指定.pvk 文件的密码。
`/po` pfxpassword	指定.pfx 文件的密码。如果未指定.pfx 文件的密码，.pfx 文件的密码将作为.pvk 文件的密码相同。
`/f`	配置 Pvk2Pfx 覆盖.pfx 文件，如果存在与指定的同名 -pfx切换。

若要使用 SignTool 工具来签署驱动程序符合的方式使用 SPC 内核模式代码签署策略，必须将 SPC 信息添加到对进行签名的驱动程序在本地计算机上个人证书存储。有关如何将 SPC 信息添加到个人证书存储区的信息，请参阅「Microsoft Docs - 软件发布者证书」。

5.4 signtool.exe

签名工具是一个命令行工具，用于对文件进行数字签名，以及验证文件和时间戳文件中的签名。

5.4.1 语法格式
在命令提示符处，键入以下内容：

signtool [command] [options] [file_name | ...]

5.4.2 参数

参数	描述
`command`	指定要对文件执行的操作的四个命令（`catdb`、`sign`、`Timestamp` 或 `Verify`）之一。有关每个命令的说明，请参见下一个表。
`options`	用于修改命令的选项。除全局 `/q` 和 `/v` 选项之外，每个命令均支持一组唯一选项。
`file_name`	要进行签名的文件的路径。

签名工具支持下列命令。每个命令均与不同的选项集结合使用，这些选项集已在其各自的节中列出。

命令	描述
`catdb`	在目录数据库中添加或移除目录文件。目录数据库用于自动查找目录文件，并由 GUID 标识。有关 `catdb` 命令支持的选项列表，请参阅 catdb 命令选项。
`sign`	对文件进行数字签名。数字签名可以阻止文件被篡改，并且使用户能够基于签名证书验证签名者。有关 `sign` 命令支持的选项列表，请参阅 sign 命令选项。
`Timestamp`	为文件添加时间戳。有关 `TimeStamp` 命令支持的选项列表，请参阅 TimeStamp 命令选项。
`Verify`	通过确定签名证书是否由受信任的颁发机构颁发、是否已撤消签名证书，以及签名证书对于特定策略是否有效（可选）来验证文件的数字签名。有关 `Verify` 命令支持的选项列表，请参阅 Verify 命令选项。

下列选项适用于所有签名工具命令。

全局选项	描述
/q	如果命令运行成功，则不显示输出；如果命令运行失败，则显示最小输出。
/v	无论命令是否运行成功，都显示详细输出，并显示警告消息。
/debug	显示调试信息。

5.4.3 catdb 命令选项
下表列出了可与 catdb 命令一起使用的选项。

Catdb 选项	描述
`/d`	指定更新默认目录数据库。如果 `/d` 和 `/g` 选项都未使用，则签名工具会更新系统组件和驱动程序数据库。
`/g` GUID	指定由全局唯一标识符 GUID 标识的目录数据库已更新。
`/r`	从目录数据库中移除指定的目录。如果未指定该选项，签名工具将向目录数据库添加指定目录。
`/u`	指定自动为添加的目录文件生成唯一名称。如有必要，重命名目录文件以阻止与现有目录文件发生名称冲突。如果未指定该选项，签名工具将覆盖与所添加的目录同名的任何现有目录。

5.4.4 sign 命令选项
下表列出了可与 sign 命令一起使用的选项。

Sign 命令选项	描述
`/a`	自动选择最佳签名证书。签名工具将查找满足所有指定条件的所有有效证书，并选择有效时间最长的证书。如果未提供该选项，签名工具仅查找一个有效的签名证书。
`/ac` file	将 file 中的其他证书添加到签名块。
`/as`	追加此签名。如果不存在主签名，则改为使此签名成为主签名。
`/c` CertTemplateName	指定用于对证书进行签名的证书模板名（一个 Microsoft 扩展）。
`/csp` CSPName	指定包含私钥容器的加密服务提供程序 (CSP)。
`/d` Desc	指定已签名内容的说明。
`/du` URL	为已签名内容的详细说明指定统一资源定位器 (URL)。
`/f` SignCertFile	指定文件中的签名证书。如果文件采用个人信息交换 (PFX) 格式且受密码保护，则使用 `/p` 选项指定密码。如果文件不包含私钥，则使用 `/csp` 和 `/kc` 选项指定 CSP 和私钥容器名。
`/fd`	指定要用于创建文件签名的文件摘要算法。默认值为 SHA1。
`/i` IssuerName	指定签名证书的颁发者的名称。该值可以是整个颁发者名称的子字符串。
`/kc` PrivKeyContainerName	指定私钥容器名。
`/n` SubjectName	指定签名证书的主题的名称。该值可以是整个主题名称的子字符串。
`/nph`	如果支持，则取消可执行文件的页面哈希。默认值由 SIGNTOOL_PAGE_HASHES 环境变量和 wintrust.dll 版本决定。对于非 PE 文件，忽略此选项。
`/p` Password	指定打开 PFX 文件时要使用的密码。（使用 `/f` 选项指定 PFX 文件。）
`/p7` 路径	指定为每个指定的内容文件生成的公钥加密标准 (PKCS) #7 文件。 PKCS #7 文件命名为 path\filename.p7。
`/p7ce` Value	为已签名的 PKCS #7 内容指定选项。将 Value 设置为“嵌入的”，可将已签名内容嵌入到 PKCS #7 文件中；如果设置为“DetachedSignedData”，则可生成分离的 PKCS #7 文件的已签名数据部分。如果未使用 `/p7ce` 选项，默认情况下将嵌入已签名的内容。
`/p7co` <OID>	指定标识已签名的 PKCS #7 内容的对象标识符 (OID)。
`/ph`	如果支持，则生成可执行文件的页面哈希。
`/r` RootSubjectName	指定签名证书必须链接到的根证书的主题名称。该值可以是根证书的整个主题名称的子字符串。
`/s` StoreName	指定要在搜索证书时打开的存储。如果未指定该选项，则打开 `My` 存储。
`/sha1` Hash	指定签名证书的 SHA1 哈希。当多个证书满足剩余开关指定的条件时，通常会指定 SHA1 哈希。
`/sm`	指定使用计算机存储，而不是用户存储。
`/t` URL	指定时间戳服务器的 URL。如果该选项（或 `/tr`）不存在，将不会对签名文件执行时间戳操作。如果时间戳操作失败，将生成一个警告。此选项不能与 `/tr` 选项一起使用。
`/td` alg	将此选项与 `/tr` 选项一起使用可请求 RFC 3161 时间戳服务器使用的摘要算法。
`/tr` URL	指定 RFC 3161 时间戳服务器的 URL。如果该选项（或 `/t`）不存在，将不会对签名文件执行时间戳操作。如果时间戳操作失败，将生成一个警告。此选项不能与 `/t` 选项一起使用。
`/u` Usage	指定签名证书中必须存在的增强型密钥用法 (EKU)。可以通过 OID 或字符串指定该用法的值。默认用法为“代码签名”(1.3.6.1.5.5.7.3.3)。
`/uw`	指定“Windows 系统组件验证”(1.3.6.1.4.1.311.10.3.6) 的用法。

有关用法示例，请参阅「Microsoft Docs - Using SignTool to Sign a File」（使用 SignTool 为文件签名）。

5.4.5 TimeStamp 命令选项 下表列出了可与 TimeStamp 命令一起使用的选项。

TimeStamp 选项	描述
`/p7`	对 PKCS #7 文件执行时间戳操作。
`/t` URL	指定时间戳服务器的 URL。要执行时间戳操作的文件必须在以前已进行签名。需要 `/t` 或 `/tr` 选项。
`/td` alg	请求 RFC 3161 时间戳服务器使用的摘要算法。 `/td` 与 `/tr` 选项一起使用。
`/tp` index	对 index 处的签名进行时间戳操作。
`/tr` URL	指定 RFC 3161 时间戳服务器的 URL。要执行时间戳操作的文件必须在以前已进行签名。需要 `/tr` 或 `/t` 选项。

有关使用示例，请参阅「Microsoft Docs - Adding Time Stamps to Previously Signed Files」（向之前已签名的文件添加时间戳）。

5.4.6 Verify 命令选项

Verify 选项	描述
`/a`	指定可以使用所有方法来验证文件。首先，搜索目录数据库以确定是否在目录中对文件进行签名。如果未在任何目录中对文件进行签名，签名工具将尝试验证文件的嵌入签名。验证可以或不能在目录中进行签名的文件时，建议使用该选项。这些文件的示例包括 Windows 文件或驱动程序。
`/ad`	使用默认的目录数据库查找目录。
`/ag` CatDBGUID	在由 CatDBGUID 标识的目录数据库中查找目录。
`/all`	验证包含多个签名的文件中的所有签名。
`/as`	使用系统组件（驱动程序）目录数据库查找目录。
`/c` CatFile	通过名称指定目录文件。
`/d`	指定签名工具应打印描述和描述 URL。
`/ds` Index	验证指定位置的签名。
`/hash` (`SHA1`\|`SHA256`)	指定在目录中搜索文件时要使用的可选哈希算法。
`/kp`	指定应使用内核模式驱动程序签名策略执行验证。
`/ms`	使用多个验证语义。这是 Windows 8 和更高版本上的「Microsoft Docs - WinVerifyTrust」调用的默认行为。
`/o` Version	按操作系统版本验证文件。版本具有以下格式：PlatformID：VerMajor.VerMinor.BuildNumber。 PlatformID 表示枚举成员的基础值。重要提示：建议使用 `/o` 开关。如果未指定 `/o`，SignTool.exe 可能会返回意外的结果。例如，如果你未将 `/o` 开关包含在内，则能在旧版操作系统上正确验证的系统目录可能在新版操作系统上无法正确验证。
`/p7`	验证 PKCS #7 文件。无现有策略用于 PKCS #7 验证。该签名处于选中状态，并为签名证书生成了链。
`/pa`	指定应使用默认认证码验证策略。如果未指定 `/pa` 选项，签名工具将使用 Windows 驱动程序验证策略。此选项不能与 `catdb` 选项一起使用。
`/pg` PolicyGUID	通过 GUID 指定验证策略。 PolicyGUID 相当于验证策略的 ActionID。此选项不能与 `catdb` 选项一起使用。
`/ph`	指定签名工具应打印并验证页面哈希值。
`/r` RootSubjectName	指定签名证书必须链接到的根证书的主题名称。该值可以是根证书的整个主题名称的子字符串。
`/tw`	指定在未对签名进行时间戳操作时应生成警告。

5.4.7 返回值
当其终止时，签名工具将返回下列退出代码之一。

退出代码	描述
0	执行成功。
1	执行失败。
2	执行完成，但出现警告。

5.4.8 示例
以下命令将目录文件 MyCatalogFileName.cat 添加到系统组件和驱动程序数据库中。如有必要阻止替换名为 /u 的现有目录文件，MyCatalogFileName.cat 选项会生成唯一名称。

signtool catdb /v /u MyCatalogFileName.cat

以下命令通过使用最佳证书对文件进行自动签名。

signtool sign /a MyFile.exe

以下命令使用存储在受密码保护的 PFX 文件中的证书对文件进行数字签名。

signtool sign /f MyCert.pfx /p MyPassword MyFile.exe

以下命令对文件进行数字签名并加盖时间戳。用于对文件进行签名的证书存储在 PFX 文件中。

signtool sign /f MyCert.pfx /t http://timestamp.digicert.com MyFile.exe

以下命令通过使用位于 My 存储中的证书对文件进行签名，该证书的主题名为 My Company Certificate。

signtool sign /n "My Company Certificate" MyFile.exe

以下命令对 ActiveX 控件进行签名，并提供在系统提示用户安装此控件时由 Internet Explorer 显示的信息。

Signtool sign /f MyCert.pfx /d: "MyControl" /du http://www.example.com/MyControl/info.html MyControl.exe

以下命令对已进行数字签名的文件加盖时间戳。

signtool timestamp /t http://timestamp.digicert.com MyFile.exe

以下命令确认文件已签名。

signtool verify MyFile.exe

以下命令验证可能已在目录中签名的系统文件。

signtool verify /a SystemFile.dll

以下命令验证已在名为 MyCatalog.cat 目录中签名的系统文件。

signtool verify /c MyCatalog.cat SystemFile.dll

Install and remove application with command line in Ubuntu

David Dong — Sun, 19 Jul 2020 11:06:44 +0800

How to install and remove software in Ubuntu? Don’t worry, Ubuntu provides many ways to assist you in this task. Some of them are graphical tools like the built-in “Ubuntu Software Center,” or well-known tools such as “Synaptic” and “gdebi.” In this post, I want to discuss apt, a command-line tool. Because I prefer command-line tools, they are fast and straightforward.

what is APT?

APT (Advanced Package Tool) is the command line tool to interact with the packaging system in Debian-based Linux distributions. There is already dpkg commands to manage it. But APT is a more friendly way to handle packaging. You can use it to find and install new packages, upgrade packages, remove the packages etc. You may heard about apt-get, but apt is new. what is the difference between apt and apt-get? In a short, apt is a subset of apt-get and apt-cache commands providing necessary commands for package management. apt is more friendly to users because it is easier to use with its fewer but easy to remember options.

Using APT commands to manage packages

some location need to know

the location to store the software (.deb package) that APT download from the PPA /var/cache/apt/archives
the location to store the softwae after install by APT

/usr/share/packagename
the path of the sources.list

/etc/apt/sources.list
the location to store the executable bin file

/usr/bin
the location to store the libs

/usr/lib

add software ppa

APT uses PPA to install software, But what is PPA? Why is it used? Is it safe to use PPA? How to properly use PPA? How to delete a PPA?

Here I would not discuss too much about it, PPA means Personal Package Archive. You can unserstand it as a URL of software repository where allows application developers and Linux users to create their own repositories to distribute software.

suggest to read PPA in Ubuntu Linux [Definitive Guide] for more details.

# add ppa into sources.list
sudo add-apt-repository ppa:khurshid-alam/nautilus-git
# remove ppa
sudo add-apt-repository --remove ppa:khurshid-alam/nautilus-git

update package databas with APT

sudo apt update

APT uses the database to manager the package version, updating the repository should be the first thing to do before the installation of package. elsewise, APT won’t know if there are any newer packages available. this command won’t upgrade the real package but just refresh the package version information in the database.

install software package

sudo apt install nautilus-compare
# install specific version package
sudo apt install nautilus-compare=0.0.6~bionic3

apt install command will install the latest version of the package in the repository. normally the repository will not contain the older version. If you have already installed the package, running the command will upgrade the package to the latest version. If you want to install the package without upgrading, try this command

sudo apt install nautilus-compare --no-upgrade

If you only want to upgrade a package but don’t want to install it (if it’s not already installed), you can do that with the following command:

sudo apt install nautilus-compare --only-upgrade

remove software package

# remove installed package
sudo apt remove nautilus-compare
# remove installed package and related configure file
sudo purge nautilus-compare
# remove package and it's dependency
sudo apt autoremove nautilus-compare

What is the difference between apt remove and apt purge?

apt remove just removes the binaries of a package. It leaves residue configuration files.
apt purge removes everything related to a package including the configuration files.

If you used apt remove to a get rid of a particular software and then install it again, your software will have the same configuration files.

search for package

sudo apt search nautilus-compare

another way to search the available package is using Launchpad website, the official platform for hosting PPA. You can go to Launchpad and search for the required package directly there.

See the content of a package

sudo apt show nautilus-compare

This will show information about the given package(s) like its dependencies, installation and download size, different sources the package is available from, the description of the content of the package among other things.

sudo apt policy nautilus-compare
# show package info and dependency
sudo apt-cache showpkg rsync
sudo apt-cache depends rsync
# show the 
sudo apt-cache rdenpends rsync

List upgradable and installed versions

sudo apt list --installed
sudo apt list --upgradable
sudo apt list --all-version

Clean the system

sudo apt autoremove
sudo apt autoremove --purge

this command removes the unnecessary libs or packages that are once as the dependency by some applications and didn’t used.

When you install a package using apt-get or apt command (or DEB packages in the software center), the apt package manager downloads the package and its dependencies in .deb format and keeps it in /var/cache/apt/archives folder.

Once the deb files for the package and its dependencies are downloaded, your system installs the package from these deb files.

if you want to clean the cache of download package in the /var/cache/apt/archieve

sudo apt-get clean
sudo apt-get autoclean

upgrade package

#auto upgrade all package
sudo apt upgrade
#upgrade os
sudo apt dist-upgrade
# only upgrade specific package
sudo apt install --only-upgrade rsync
# or run insall command to upgrade to latest version
sudo apt install rsync

Hold the package and prevent upgrade

If you manage to install a specific program version, you may want to avoid accidentally upgrading to the newer version. It’s not too complicated to achieve this.

# lock package version, don't upgrade by apt upgrade command
sudo apt-mark hold rsync

You can remove the hold so that it can be upgraded later:

sudo apt-mark unhold rsync

set manual install

sudo apt-mark manual rsync

sudo apt-mark showhold
sudo apt-mark showauto
sudo apt-mark showmanual

fix the install

sudo apt -f install

this command will fix the dependency broken error durring the installtion.

standalone .deb package install and remove

# download deb package
sudo apt download nautilus-compare
# install by dpkg
sudo dpkg -i nautilus-compare_1.0.0~focal1_all.deb

#remove 
sudo dpkg -r nautilus-compare_1.0.0~focal1_all.deb
sudo dpkg --remove --purge nautilus-compare_1.0.0~focal1_all.deb

#list
sudo dpkg -l rsync
# show the deb package contents
sudo dpkg -c nautilus-compare_1.0.0~focal1_all.deb
# show the package installed files in the system
sudo dpkg -L copyq
# find the file belongs to which package
sudo dpkg -S /usr/bin/rsync

apt-key manager GPG key

You can see the GPG keys added by various repositories in your system using the apt-key list command.

# show key 
sudo apt-key list

add key

sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys E298A3A825C0D65DFD57CBB651716619E084DAB9

delete key

sudo apt-key del 12345

apt lock

you may encounter the issue when you use APT tool install package, as below

you cannot acquire the lock /var/lib/dpkg/lock-fronted.

There are 3 lock files in the ubuntu that used for apt/dpkg

/var/lib/dpkg/lock-frontend
/var/cache/apt/archieve/lock
/var/lib/dpkg/lock

one way to fix the lock issue is running the command to remove the lock file.

sudo rm -rf /var/lib/dpkg/lock-frontend /var/cache/apt/archieve/lock /var/lib/dpkg/lock

you may not have to remove all of them, just find which lock file cannot be acquired and delete it!

mbedtls 使用说明

David Dong — Fri, 10 Jul 2020 10:24:12 +0800

最近的工作中使用到了mbedtls，这里做一个简单的总结。

1. 什么是 mbedtls

mbedtls 一句话来概况的话就是它是一个开源的SSL/TLS库。
–项目源码地址
 –项目API文档

1.1 SSL

关于SSL(Secure Sockets Layer 安全套接字协议)，百度百科的解释如下：

SSL协议位于TCP/IP协议与各种应用层协议之间，为数据通讯提供安全支持。SSL协议可分为两层：

SSL记录协议（SSL Record Protocol）：它建立在可靠的传输协议（如TCP）之上，为高层协议提供数据封装、压缩、加密等基本功能的支持。

SSL握手协议（SSL Handshake Protocol）：它建立在SSL记录协议之上，用于在实际的数据传输开始前，通讯双方进行身份认证、协商加密算法、交换加密密钥等。

TLS与SSL在传输层与应用层之间对网络连接进行加密。

1.2 mbedtls 特点

mbedtls 有以下的特点：

简洁的API
开源代码库
功能模块化，松耦合。
模块可单独配置，编译
C 语言编写，针对嵌入式应用

从功能角度来看，mbedtls可以分为三个主要部分：

SSL/TLS 协议库（libmbedtls.a）
加密库（libmbedcrypto.a）
X.509 证书处理库（libmbedx509.a）

注意：这三个库在生成的时候的依赖关系，libmbedtls.a 依赖 libmbedx509.a 而 libmbedx509.a 依赖 libmbedcrypto.a。 因此在生成时要注意顺序问题。

1.3 mbedtls 模块

mbedtls库包含以下的模块。

模块	说明
Encryption/decryption module	加解密模块
Hashing module	The Message Digest (MD)提供了hash 功能
Random number generator (RNG) module	提供了随机数产生功能，see mbedtls_ctr_drbg_random()
SSL/TLS communication module	提供了方式建立SSL/TLS通讯通道
TCP/IP communication module	provides for a channel of communication for the SSL/TLS communication module to use
X.509 module	提供了X.509的支持，用于证书的读写和认证
Asn1_module

2. mbedtls 安装与配置

mbedtls支持3种 build 方式，

GNU Make
CMake
Microsoft Visual Studio (Microsoft Visual Studio 2013 or later)

2.1 工具链支持

GNU Make 或者 CMake build 工具.
C99 工具链 (compiler, linker, achiever). 目前支持 GCC5.4, Clang 3.8, IAR8 和 Visual Studio 2013 以后的版本.
Python 3 用于产生测试代码.
Perl 用于运行测试.

2.2 GNU Make

run build

make

build with test

make check

build without test

make no_test

2.3 CMake

run build

mkdir /path/to/build_dir && cd /path/to/build_dir
cmake /path/to/mbedtls_source
cmake --build .

run test

ctest

build without test

cmake -DENABLE_TESTING=Off /path/to/mbedtls_source

make shared library

cmake -DUSE_SHARED_MBEDTLS_LIBRARY=On /path/to/mbedtls_source

2.4 Microsoft Visual Studio

工程文件mbedTLS.sln 包含了所有必需的项目和程序. 编译该工程文件可以生成库文件。

关于更多的相关信息，可以参阅 GitHub README for Mbed TLS。

3. mbedtls 使用

mbedtls通过提供以下功能支持SSLv3到TLSv1.2版本的通信：

TCP/IP通信功能：监听、连接、接受、读/写
SSL/TLS通信功能：初始化、握手、读/写
X.509功能：CRT、CRL和按键处理
随机数生成
散列
加密/解密

上面的功能被整齐地设计成逻辑接口。它们可以单独用于提供上述任何功能，或者混合和匹配到使用X.509 PKI的SSL服务器/客户端解决方案中。

3.1 例子

服务器端的设定:
先决条件

X.509证书和私钥
会话处理函数 设定步骤
加载证书和私钥（X.509接口）
设置侦听TCP套接字（TCP/IP接口）
接受传入客户端连接（TCP/IP接口）
初始化为SSL服务器（SSL/TLS接口）
设置参数，例如身份验证、密码、CA链、密钥交换
设置回调函数RNG、IO、会话处理
执行SSL握手（SSL/TLS接口）
读/写数据（SSL/TLS接口）
关闭和清理（所有接口）

客户端的设定:
先决条件

X.509证书和私钥
X.509可信CA证书 设定步骤
加载受信任的CA证书（X.509接口）
加载证书和私钥（X.509接口）
设置TCP/IP连接（TCP/IP接口）
初始化为SSL客户端（SSL/TLS接口）
设置参数，例如身份验证模式、密码、CA链、会话
设置回调函数RNG，IO
执行SSL握手（SSL/TLS接口）
验证服务器证书（SSL/TLS接口）
写入/读取数据（SSL/TLS接口）
关闭和清理（所有接口）

3.2 SSL/TLS 部分功能

mbed tls的ssl/tls部分提供了使用ssl/tls通过安全通信通道建立和通信方法。

其基本规定是:

初始化一个SSL/TLS context
执行一个SSL/TLS握手(handshake)
发送/接收数据
通知对方一个连接正在关闭

一个通道很多方面都是通过参数和回调函数来设置的 »>

端点角色(endpoint role),客户端和服务器
身份验证模式: 是否应该进行证书验证
主机到主机通信通道: 发送和接收功能
随机数生成器(RNG)功能
用于加密/解密的密码
证书验证功能
会话控制: 会话获取和设置功能
证书处理和密钥交换的X.509参数

mbed 通过创建一个SSL/TLS服务器和客户端,通过提供一个框架来建立和通过SSL/TLS通信通道进行通信.SSL/TLS部分直接依赖于库的证书解析,对称和非对称和哈希模块.

3.3 数据结构

mbedtls_net_context：目前只有文件描述符，可以用于适配异步I/O库
mbedtls_ssl_context：保存SSL基本数据
mbedtls_ssl_config: SSL 配置数据
mbedtls_ctr_drbg_context
mbedtls_entropy_context：保存熵配置
mbedtls_x509_crt：保存认证信息

3.4 Init 阶段

下面时init阶段需要调用的函数与传统 socket 的对比。传统的socket-based的程序，依照顺序，作为client要做以下的函数调用：

gethostbyname()
socket()
connect()
write()
read()

改成SSL之后，mbedTLS对应上述函数，分别对应为：

gethostbyname()   \ 
socket()          -+--> mbedtls_net_connect() + mbedtls_ssl_handshake()
connect()         /
write()           ----> mbedtls_ssl_write()
read()            ----> mbedtls_ssl_read()

当然，实际情况下，会使用更多的其他函数。

下面是init阶段需要调用的各函数。函数的参数，在调用的时候按照上面的函数类型一个一个传入就行了。

mbedtls_net_init()
mbedtld_ssl_init()
mbedtld_ssl_config_init()
mbedtls_ctr_drbg_init()
mbedtld_x509_crt_init()
mbedtls_entropy_init()
mebdtls_ctr_drbg_seed()

其中mebdtls_ctr_drbg_seed()可以指定熵函数。如果回调使用默认的mbedtls_entropy_func的话，可以传入一个初始的熵seed，也可以NULL。

3.5 Connect 阶段

mbedtls_net_connect()：参数是server和端口，均为字符串。server可以使域名或者IP字符串。最后一个参数使用MBEDTLS_NET_PROTO_TCP即可。端口号不仅仅可以传入数字字符串，也可以类似于get_addrinfo函数的protocol参数那样，传入类似于HTTPS这样的可读化字符串。

Inf2Cat 工具

David Dong — Thu, 09 Jul 2020 18:24:12 +0800

Windows 驱动开发中经常会用到 Windows WDK 自带的 Inf2Cat.exe 工具将驱动程序的 INF 文件转成待签名的 CAT 格式的目录文件。Windows 7 64位系统以上的版本，如果没有对cat或者sys文件进行数字签名，驱动安装会出现问题，或者驱动使用过程也会出现异常。因此，对驱动程序的签名已经是一项必须的操作，这里就介绍一下 Inf2Cat.exe 这个工具。这个工具本身并不能生成签名，但是却是驱动程序签名的必要的一步。

Inf2Cat

MSDN里对这个工具的说明如下：

Inf2Cat (Inf2Cat.exe) is a command-line tool that determines whether a driver package’s INF file can be digitally-signed for a specified list of Windows versions. If so, Inf2Cat generates the unsigned catalog files that apply to the specified Windows versions.

Inf2Cat 工具检查驱动程序包的 INF 文件，了解是否存在结构错误，并且是否可以对验证驱动程序包进行数字签名。仅当 INF 文件中引用的所有文件都存在且源文件位于正确的位置时，才能对驱动程序包进行签名。

工具路径

c:\Program Files (x86)\Windows Kits\10\bin\x86

使用方法

Inf2Cat /driver:PackagePath /os:WindowsVersionList [/nocat] [/verbose] [/?] [other switches]

参数说明

参数	说明
/driver:PackagePath	指定包含驱动程序包的 INF 文件的目录路径。如果指定的目录包含多个驱动程序包的 INF 文件，那么 Inf2Cat 会为每个驱动程序包创建目录文件
/nocat	将 Inf2Cat 配置为验证驱动程序包是否符合指定的 Windows 版本的签名要求，而不生成目录文件
/os:WindowsVersionList	将 Inf2Cat 配置为验证驱动程序包的 INF 文件是否符合由 WindowsVersionList 指定的 Windows 版本的签名要求。WindowsVersionList 是一个逗号分隔列表，其中可以包含一个或多个版本标识符
/verbose	将 Inf2Cat 配置为在命令窗口中显示详细信息
/?	将 Inf2Cat 配置为在命令窗口中显示帮助信息
/uselocaltime	运行驱动程序时间戳验证测试时使用本地时区

Windows 版本标识符

Windows 版本	版本标识符
Windows 10 x86 19H1 Edition	10_19H1_X86
Windows 10 x64 19H1 Edition	10_19H1_X64
Windows 10 ARM64 19H1 Edition	10_19H1_ARM64
Windows 10 x86 RS5 Edition	10_RS5_X86
Windows 10 x64 RS5 Edition	10_RS5_X64
Windows 10 ARM64 RS5 Edition	10_RS5_ARM64
Windows Server RS5 x64 Edition	ServerRS5_X64
Windows Server RS5 ARM64 Edition	ServerRS5_ARM64
Windows 10 x86 RS4 Edition	10_RS4_X86
Windows 10 x64 RS4 Edition	10_RS4_X64
Windows 10 ARM64 RS4 Edition	10_RS4_ARM64
Windows 10 x86 RS3 Edition	10_RS3_X86
Windows 10 x64 RS3 Edition	10_RS3_X64
Windows 10 ARM64 RS3 Edition	10_RS3_ARM64
Windows 10 x86 RS2 Edition	10_RS2_X86
Windows 10 x64 RS2 Edition	10_RS2_X64
Windows 10 x86 RS1 Edition	10_AU_X86
Windows 10 x64 RS1 Edition	10_AU_X64
Windows Server 2016 x64 Edition	SERVER2016_X64
Windows 10 x86 Edition	10_X86
Windows 10 x64 Edition	10_X64
Windows Server 2016	Server10_X64
Windows Server 2016 on ARM	Server10_ARM64
Windows 8.1 x86 Edition	6_3_X86
Windows 8.1 x64 Edition	6_3_X64
Windows 8.1 ARM Edition	6_3_ARM
Windows Server 2012 R2	Server6_3_X64
Windows 8 x64 Edition	8_X64
Windows 8 x86 Edition	8_X86
Windows 8 ARM Edition	8_ARM
Windows Server 2012	Server8_X64
Windows Server 2008 R2 x64 Edition	Server2008R2_X64
Windows Server 2008 R2 Itanium Edition	Server2008R2_IA64
Windows 7 x64 Edition	7_X64
Windows 7 x86 Edition	7_X86
Windows Server 2008 x64 Edition	Server2008_X64
Windows Server 2008 Itanium Edition	Server2008_IA64
Windows Server 2008 x86 Edition	Server2008_X86
Windows Vista x64 Edition	Vista_X64
Windows Vista x86 Edition	Vista_X86
Windows XP x64 Edition	XP_X64
Windows XP x86 Edition	XP_X86
Windows Server 2003 x64 Edition	Server2003_X64
Windows Server 2003 Itanium Edition	Server2003_IA64
Windows Server 2003 x86 Edition	Server2003_X86

Inf2Cat 忽略版本标识符字符串的字母字符的大小写。例如，vista_x64 和 Vista_X64 对于 Windows Vista x64 版本来说都是有效的标识符。

只有在驱动程序包的INF文件中指定了目录文件并且目录文件适用于一个或多个指定的Windows版本时，Inf2Cat才会生成目录文件。如果INF文件的[INF Version section]部分只提供一个目录文件=文件名.cat指令，则该目录文件将应用于整个驱动程序包。

应用示例

在以下示例中，c:\MyDriver 包含一个驱动程序包，该程序包的 INF 文件为 MyInfFile.inf 并且 INF 文件中的 INF 版本部分仅包含以下 CatalogFile 指令：

[Version]
...
CatalogFile=MyCatalogFile.cat
...

对于该示例，以下 Inf2Cat 命令将验证是否可以针对 Windows 2000 和 x86 版本的 Windows Vista、Windows Server 2003 和 Windows XP 对驱动程序包进行签名。如果可以针对这些版本对程序包进行签名，那么 Inf2Cat 将创建未签名的目录文件 MyCatalogFile.cat。

Inf2Cat /driver:C:\MyDriver /os:2000,XP_X86,Server2003_X86,Vista_X86

在以下示例中，c:\MyDriver 包含一个驱动程序包，该程序包的 INF 文件为 MyInfFile.inf，并且 INF 文件中的 INF Version 部分仅包含以下两个具有平台扩展的CatalogFile 指令：

[Version]
...
CatalogFile.ntx86=MyCatalogFileX86.cat
CatalogFile.ntamd64=MyCatalogFileX64.cat
...

对于该示例，以下 Inf2Cat 命令将验证是否可以针对 Windows 2000 和 x86 版本的 Windows Vista、Windows Server 2003 和 Windows XP 对驱动程序包进行签名。此外，该命令还将验证是否可以针对 x64 版本的 Windows Vista、Windows Server 2003 和 Windows XP 对驱动程序包进行签名。如果可以针对所有这些版本对程序包进行签名，那么 Inf2Cat 将创建未签名的目录文件 MyCatalogFileX86.cat 和 MyCatalogFileX64.cat。

Inf2Cat /driver:C:\MyDriver /os:2000,XP_X86,XP_X64,Server2003_X86,Server2003_X64,Vista_X86,Vista_X64

浅谈数字签名

David Dong — Wed, 01 Jul 2020 18:24:12 +0800

近日，腾讯公司状告贵州老干妈的新闻在互联网上闹得沸沸扬扬，💥 大家也应该有所耳闻。腾讯起诉老干妈合作推广后没有付广告费，但随后老干妈否认了此事，称完全不知道这个合作。 😕 后来经调查发现，这是三个人冒充老干妈市场部经理，然后刻了个假公章，就为了获得合作后的游戏礼包并从中牟利。

且不说这个事情的性质，就从三个骗子用一个假公章，居然把腾讯骗得团团转来看，也折射出来目前的一些问题。

试想如果腾讯在和老干妈的合同中使用了数字签名… 上述事件根本就不会发生。

这篇文章就来聊一聊数字签名。文章比较长，心急的读者可以选择直接跳到相关章节阅读。

本文就从散列函数开始说起，然后引出 👉 数字摘要技术和非对称加密技术，最后介绍基于这两个技术应用的数字签名。

1. 数字摘要技术

1.1 散列函数

官方解释：散列函数一般是指Hash函数，是一种从任何一种数据中创建小的数字指纹的方法。散列函数把消息或数据压缩成摘要，使得数据量变小，将数据的格式固定下来。

我的理解：散列函数是将任意长度的消息映射成为定长的散列值的函数，以该散列值（消息摘要）作为认证符。

1.2 密码散列函数

密码散列函数（Cryptographic hash function），又译为加密散列函数，是散列函数的一种。这种散列函数的输入数据，通常被称为消息（message），而它的输出结果，经常被称为消息摘要（message digest）或摘要（digest）。将任意长度的消息压缩到某一固定长度的消息摘要，一个理想的密码散列函数应该有四个主要的特性：

对于任何一个给定的消息，它都很容易就能运算出散列数值（摘要）。
难以由一个已知的散列数值 (摘要)，去推算出原始的消息。（评注：单向，不可逆）
在不更动散列数值 (摘要) 的前提下，修改消息内容是不可行的。（评注：唯一性）
对于两个不同的消息，它不能给与相同的散列数值 (摘要)。

单向散列函数生成的信息摘要是不可预见的，消息摘要看起来和原始的数据没有任何的关系。而且，原始数据的任何微小变化都会对生成的信息摘要产生很大的影响。

1.3 MD5

MD5信息摘要算法（MD5， Message-Digest Algorithm），一种被广泛使用的密码散列函数，可以产生出一个128位（16字节）的散列值（hash value），用于确保信息传输完整一致。

应用：

用于密码管理 - 把用户的密码以MD5值（或类似的其它算法）的方式保存起来，用户注册的时候，系统是把用户输入的密码计算成 MD5 值，然后再去和系统中保存的 MD5 值进行比较，如果密文相同，就可以认定密码是正确的，否则密码错误。通过这样的步骤，系统在并不知道用户密码明码的情况下就可以确定用户登录系统的合法性。
电子签名 - 使用 MD5算法就可以为任何文件（不管其大小、格式、数量）产生一个独一无二的数字指纹，借助这个数字指纹，通过检查文件前后 MD5 值是否发生了改变，就可以知道源文件是否被改动。

1.4 SHA

安全散列算法（SHA, Secure Hash Algorithm）是一个密码散列函数家族，是FIPS所认证的安全散列算法。SHA家族的五个算法，分别是SHA-1、SHA-224、SHA-256、SHA-384，和SHA-512，后四者有时并称为SHA-2。SHA在许多安全协定中广为使用，包括TLS和SSL、PGP、SSH、S/MIME和IPsec，曾被视为是MD5（更早之前被广为使用的杂凑函数）的后继者。

1.5 HMAC

HMAC是密钥相关的哈希运算消息认证码（Hash-based Message Authentication Code）的缩写,它可以与任何迭代散列函数捆绑使用。HMAC算法是一种基于密钥的报文完整性的验证方法，其安全性是建立在Hash加密算法基础上的。它要求通信双方共享密钥、约定算法、对报文进行Hash运算，形成固定长度的认证码。通信双方通过认证码的校验来确定报文的合法性。 HMAC算法可以用来作加密、数字签名、报文验证等。
HMAC是一种利用密码学中的散列函数来进行消息认证的一种机制，所能提供的消息认证包括两方面内容：

① 消息完整性认证：能够证明消息内容在传送过程没有被修改。
② 信源身份认证：因为通信双方共享了认证的密钥，接收方能够认证发送该数据的信源与所宣称的一致，即能够可靠地确认接收的消息与发送的一致。

1.6 消息认证码

消息认证（message authentication）就是验证消息的完整性，当接收方收到发送方的报文时，接收方能够验证收到的报文是真实的和未被篡改的。

消息认证所用的摘要算法与一般的对称或非对称加密算法不同，它并不用于防止信息被窃取，而是用于证明原文的完整性和准确性，也就是说，消息认证主要用于防止信息被篡改。

消息认证码（Message authentication code，缩写为MAC）又译为消息鉴别码、文件消息认证码、讯息鉴别码、信息认证码，是经过特定算法后产生的一小段信息，检查某段消息的完整性，以及作身份验证。

它可以用来：

检查消息的完整性
鉴别消息的来源

消息认证码的算法中，通常会使用带密钥的散列函数（HMAC），或者块密码的带认证工作模式（如CBC-MAC）。信息鉴别码不能提供对信息的保密，若要同时实现保密认证，同时需要对信息进行加密。

以上为数字摘要技术的内容，接下来我们讲一讲非对称加密技术。

2. 非对称加密技术

非对称加密算法需要两个密钥：公开密钥（public key:简称pk）和私有密钥（Secret key:简称sk）。公钥与私钥是一对，如果用公钥对数据进行加密，只有用对应的私钥才能解密。

非对称加密算法实现机密信息交换的基本过程是：甲方生成一对密钥并将公钥公开，需要向甲方发送信息的其他角色(乙方)使用该密钥(甲方的公钥)对机密信息进行加密后再发送给甲方；甲方再用自己私钥对加密后的信息进行解密。甲方想要回复乙方时正好相反，使用乙方的公钥对数据进行加密，同理，乙方使用自己的私钥来进行解密。另一方面，甲方可以使用自己的私钥对机密信息进行签名后再发送给乙方；乙方再用甲方的公钥对甲方发送回来的数据进行验签。

2.1 RSA

RSA是一种公钥密码算法，被用于公钥密码和数字签名。RSA是一种非对称加密技术。

RSA密钥至少为500位长，一般推荐使用1024位。这就使加密的计算量很大。为减少计算量，在传送信息时，常采用传统加密方法与公开密钥加密方法相结合的方式，即信息采用改进的DES或IDEA对话密钥加密，然后使用RSA密钥加密对话密钥和信息摘要。对方收到信息后，用不同的密钥解密并可核对信息摘要。

RSA算法的保密强度随其密钥的长度增加而增强。但是，密钥越长，其加解密所耗用的时间也越长。速度一直是RSA的缺陷。一般来说只用于少量数据加密。RSA的速度比对应同样安全级别的对称密码算法要慢很多。

通过上面的介绍，我们可以总结出RSA的特性:

既可以用公钥加密然后私钥解密，也可以用私钥加密然后公钥解密

公钥加密然后私钥解密，可以用于通信中拥有公钥的一方向拥有私钥的另一方传递机密信息，不被第三方窃听。可应用在原文的加密。

那么私钥加密然后公钥解密是用在什么场合呢？就是数字签名（摘要的加密）。

3. 数字签名

数字签名（又称公钥数字签名）是只有信息的发送者才能产生的别人无法伪造的一段数字串，这段数字串同时也是对信息的发送者发送信息真实性的一个有效证明。

通过发送数字签名，
接收者可以

确认接收到的消息（报文）的完整性，发送过程中有没有被篡改
鉴别发送者的身份
防抵赖

发送者可以

加密信息
防止伪造信息
防止重放攻击

数字签名应用非对称密钥加密技术与数字摘要技术。

数字签名技术是将摘要信息用发送者的私钥加密，与原文一起传送给接收者。接收者用自己的公钥解密被加密的摘要信息，然后用HASH函数对收到的原文产生一个摘要信息，与解密的摘要信息对比。如果相同，则说明收到的信息是完整的，在传输过程中没有被修改，否则说明信息被修改过，因此数字签名能够验证信息的完整性。数字签名是个加密的过程，数字签名验证是个解密的过程。

如果签名者想要对某个文件进行数字签名，公钥必须向CA (身份认证机构）来注册。注册后身份认证机构给你发一数字证书。对文件签名后，你把此数字证书连同文件及签名一起发给接受者，接受者向身份认证机构求证是否真地是用你的密钥签发的文件。

3.1 签名过程

在正式的数字签名中，发送方首先对发送文件采用哈希算法，得到一个固定长度的消息摘要(Message Digest)；
再用自己的私钥(Secret key，SK)对消息摘要进行签名，形成发送方的数字签名。
数字签名将作为队件和原文一起发送给接收方；
接收方首先用发送方的公钥对数字签名进行解密得到发送方的数字摘要，然后用相同的哈希函数对原文进行哈希计算，得到一个新的消息摘要，最后将消息摘要与收到的消息摘要做比较。

3.1.1 例子
一个完整的签名例子如下。

为了保证信息传送的保密性、真实性、完整性和不可否认性，需要对传送的信息进行数字加密和签名，其传送过程为：

发送者准备好要传送的数字信息（明文）

发送者对数字信息进行哈希运算（SH256/MD5），得到一个信息摘要；

发送者用自己的私钥对信息摘要进行加密得到数字签名（RSA），并将其附在数字信息上；

发送者随机产生一个加密密钥，并用此密码对要发送的信息（明文）进行加密（对称加密DES），形成密文；

发送者用接收者的公钥对刚才随机产生的加密密钥🔑进行加密，将加密后的DES密钥连同密文一起传送给接收者；

接收者收到传送来的密文和加密过的DES密钥，先用自己的私钥对加密的DES密钥进行解密，得到发送者随机产生的加密密钥；

接收者然后用随机密钥🔑对收到的密文进行解密，得到明文的数字信息，然后将随机密钥抛弃；

接收者用发送者的公钥对发送者的数字签名进行解密，得到信息摘要；

接收者用相同的哈希算法对收到的明文再进行一次哈希运算，得到一个新的信息摘要；

接收者将收到的信息摘要和新产生的信息摘要进行比较，如果一致，说明收到的信息没有被修改过，并且确实来自于发送者。

3.2 X.509

最后我们来说一下数字签名中用于公钥的数字证书，为了证明公钥的有效性和合法性，信息发送者必须向CA (身份认证机构）来注册。注册后身份认证机构给你发一数字证书。信息发送者在签名时必须将数字证书连同文件及签名一起发给接受者，接受者接收到信息后会通过数字证书确认是否手里的公钥为发送者的。

X.509正是一种数字证书的格式标准。

3.2.1 格式
X.509规定了数字证书的格式。
大致如下：

版本号
序列号
签名算法
颁发者
证书有效期
  开始日期
  终止日期
主题
主题公钥信息
  公钥算法
  主体公钥
颁发者唯一身份信息（可选）
主题唯一身份信息（可选）
扩展信息（可选）
签名

3.2.2 扩展名
X.509有多种常用的扩展名。不过其中的一些还用于其它用途，就是说具有这个扩展名的文件可能并不是证书，比如说可能只是保存了私钥。

扩展名	说明
.pem(隐私增强型电子邮件)	DER编码的证书再进行Base64编码的数据存放在`-----BEGIN CERTIFICATE-----`和`-----END CERTIFICATE-----`之中
.cer/.crt/.der	通常是DER二进制格式的，但Base64编码后也很常见。
p7b/.p7c	PKCS
.p12	PKCS#12格式，包含证书的同时可能还有带密码保护的私钥。
.pfx	PKCS#12之前的格式（通常用PKCS#12格式，比如那些由IIS产生的PFX文件）。

3.3 SSL证书

符合X.509格式的证书有多种，这里以SSL证书为例。

常见的场景会是下面这样：

客户端，比如浏览器会向服务端发送请求，服务端为了证明自己的身份，会发送证书给对方。

浏览器读取证书的数字签名部分，用自身根证书列表中对应的公钥证书对其进行解密。如果解密成功，并且证书哈希值与签名内的哈希值匹配一致，可证明站点提供的证书确实是该CA根证书签发的，否则给出风险提示。

验证通过之后，使用证书中的公钥对随机数和对称加密算法加密，发送给服务端，服务端用私钥进行解密，获得密钥和加密算法。

服务端与浏览器后续通信将会使用新的对称加密算法和随机密钥加密信息。

SSL证书从生成到使用涉及到了三次加解密过程：

证书生成的时候利用私钥签名，验证证书的时候利用公钥解密。
确认证书有效后，利用证书中的公钥进行加密，服务端利用私钥解密。
双方使用新生成的随机密钥进行数据加解密。

用一张结构图表示下HTTPS和X.509的关系：

    HTTPS
  /       \
HTTP    TLS/SSL
        /      \
   通信内容    确认身份
   对称加密    SSL证书 —— X.509 格式，非对称加密

3.4 数字签名应用

网站认证
最常见的用处就是用来认证一个网站的身份,用浏览器打开任何一个网站，然后点击地址栏前面的 🔒，点击查看证书，就可以看到网站的数字签名证书了。看过之前的内容我们就知道数字证书其实是对公钥的封装，在公钥的基础上添加了诸如颁发者之类的信息。

签名算法一栏可以看到，它使用的是sha256RSA，也就是使用SHA-256计算摘要，然后使用RSA对摘要进行签名。

代码签名
而除此之外，还有个地方我们经常碰到数字签名的——代码签名。
代码签名用于确保其来源可靠且未被篡改。
比特币
比特币是一种完全匿名的数字货币，它的身份认证是基于ECDSA。比特币的账户地址就是对公钥计算摘要得到的，向全世界公布。而确认你是账户拥有者的唯一办法就是看你有没有账户对应的私钥。对于比特币中的任意一个交易记录，只有当其中付款方的签名是有效的，它才是有效的。

3.5 生成数字签名

在Windows平台下，Microsoft提供了一组工具用于生成和校验数字签名。关于这部分内容，我会另外写一篇文章介绍如何利用 Microsoft WDK 工具来生成数字签名。

最后推荐一篇文章，讲的比我好，如果看了以上内容还有不明白的地方，可以继续阅读这篇文章。

数字签名是什么？来自：阮一峰的网络日志

COVID-19

David Dong — Sat, 20 Jun 2020 18:24:12 +0800

It has been 196 days passed since the first case was reported at WUHAN city, China. During the past half-year, millions of people were infected and more than 400000 died around the whole world. 😱

When will we end this pandemic? Do we have to co-exist with this virus in a long term? What is the world like after the epidemic?

Who knows? But I believe we will win the battle, the human beings will keep living on this planet, just like what they have done for hundreds of thousands of years.

God bless! I am lucky. In china :cn:, the situation is under control quickly after short chaos at the beginning. People around me, my family, and my friends are all healthy, no one was infected.

I hope the epidemic will eventually pass and we will return to normal life soon! ✊

🙏🏻

2020.06.20

Fingerprint driver development in Windows(3)

David Dong — Sat, 13 Jun 2020 10:37:55 +0800

Following the last two articles…

7. Managing Queues in a WBDI Driver

WBDI drivers should create at least one queue to handle multiple concurrent requests from the service. If you are using UMDF, you can take advantage of its queue management support.

In WudfBioUsbSample, the CBiometricIoQueue class implements the I/O queue interface.

In the method CBiometricIoQueue::Initialize, specifically, the driver queries the owning CBiometricIoQueue object for a pointer to the IQueueCallbackDeviceIoControl interface that the framework uses to determine the event callback functions that the driver subscribes to on the queue:

if (SUCCEEDED(hr)) 
{
    hr = this->QueryInterface(__uuidof(IUnknown), (void **)&unknown);
}

Then the driver calls IWDFDevice::CreateIoQueue to configure the default I/O queue:

hr = FxDevice->CreateIoQueue(unknown,FALSE,WdfIoQueueDispatchParallel,FALSE,FALSE,&fxQueue);
BiometricSafeRelease(unknown);

The call specifies WdfIoQueueDispatchParallel so that the framework will present requests to the driver’s I/O queue callback functions as soon as the requests are available.

Next, the driver calls IWDFDevice::ConfigureRequestDispatching to configure the queue to filter all Device I/O requests:

hr = FxDevice->ConfigureRequestDispatching(fxQueue,WdfRequestDeviceIoControl,TRUE);

Because the driver specifies WdfRequestDeviceIoControl in this call, it provides an OnDeviceIoControl handler to process I/O notifications from the framework. It does this in the IQueueCallbackDeviceIoControl::OnDeviceIoControl method that is part of the “unknown” parameter in the call to CreateIoQueue previously.

There can only be one outstanding IOCTL_BIOMETRIC_CAPTURE_DATA request at a time. The driver should track IOCTL_BIOMETRIC_CAPTURE_DATA requests, either by internally keeping a pointer to the pending requests or by using another framework queue to handle those requests.

In the sample, if there is a pending I/O request, the sample maintains a pointer to the request in a member of the CBiometricDevice class, as defined in Device.h:

IWDFIoRequest *m_PendingRequest;

While one sensor data collection I/O is pending, subsequent calls to the data collection IOCTLs should fail:

FxRequest->Complete(WINBIO_E_DATA_COLLECTION_IN_PROGRESS);

When a capture request is completed or canceled, this value is set to NULL:

IWDFIoRequest *FxRequest = (IWDFIoRequest *)InterlockedExchangePointer((PVOID *)&m_PendingRequest, NULL);

8. Installing a Biometric Driver

Vendors can provide an INF file to install a WBDI driver.

The following is a list of guidelines for biometric device installation. The code examples in this topic are taken from the WudfBioUsbSample.inx file of the WudfBioUsbSample:

WBDI drivers should specify a class of “Biometric.” Set ClassGuid equal to the value that corresponds to GUID_DEVCLASS_BIOMETRIC in Devguid.h:

[Version]
Signature="$Windows NT$"
Class=Biometric
ClassGuid={53D29EF7-377C-4D14-864B-EB3A85769359}

In your .HW section, provide AddReg directives to specify three sections that contain entries to be added to the registry:

[Biometric_Install.NT.hw]
AddReg=Biometric_Device_AddReg
AddReg=DriverPlugInAddReg, DatabaseAddReg

Provide the named sections referred to in the.HW section. The [Biometric_Device_AddReg] section sets values for the biometric device, including the exclusive flag and system wake/device idle. To be recognized by Windows Biometric Framework, UMDF-based WBDI drivers must set the “Exclusive” value to 1. The first two lines of the [Biometric_Device_AddReg] section specify access control list (ACL) rights so that the device can only be opened by an administrator or the local system account. When you specify these ACL rights, third-party applications cannot open the device and capture fingerprint data when the WinBio service is not running. For example:

[Biometric_Device_AddReg]
HKR,,"DeviceCharacteristics",0x10001,0x0100     ; Use same security checks on relative opens
HKR,,"Security",,"D:P(A;;GA;;;BA)(A;;GA;;;SY)"  ; Allow generic-all access to Built-in administrators and Local system
HKR,,"LowerFilters",0x00010008,"WinUsb" ; FLG_ADDREG_TYPE_MULTI_SZ | FLG_ADDREG_APPEND
HKR,,"Exclusive",0x10001,1
HKR,,"SystemWakeEnabled",0x00010001,1
HKR,,"DeviceIdleEnabled",0x00010001,1
HKR,,"UserSetDeviceIdleEnabled",0x00010001,1
HKR,,"DefaultIdleState",0x00010001,1
HKR,,"DefaultIdleTimeout",0x00010001,5000

A WBDI driver that exposes functionality to a legacy (non-WBDI) biometric stack should set the Exclusive value to zero. If this value is set to zero, the Windows Biometric Framework does not attempt to control the device and the device is not exposed through WBF.

Vendors can have a single driver binary that can work with legacy stacks and WBF, but the two cannot operate simultaneously. WBF will only operate if the device can be opened with exclusive access.

The second named section contains registry values for the plug-in adapters. The sample uses the Microsoft-provided sensor adapter and storage adapter. This section also enables Windows log-in support by setting the SystemSensor value:

[DriverPlugInAddReg]
HKR,WinBio\Configurations,DefaultConfiguration,,"0"
HKR,WinBio\Configurations\0,SensorMode,0x10001,1                                ; Basic - 1, Advanced - 2
HKR,WinBio\Configurations\0,SystemSensor,0x10001,1                              ; UAC/Winlogon - 1
HKR,WinBio\Configurations\0,SensorAdapterBinary,,"WinBioSensorAdapter.DLL"      ; Windows built-in WBDI sensor adapter.
HKR,WinBio\Configurations\0,EngineAdapterBinary,,"EngineAdapter.DLL"            ; Vendor engine
HKR,WinBio\Configurations\0,StorageAdapterBinary,,"WinBioStorageAdapter.DLL"    ; Windows built-in storage adapter
HKR,WinBio\Configurations\0,DatabaseId,,"6E9D4C5A-55B4-4c52-90B7-DDDC75CA4D50"  ; Unique database GUID

Finally, the third section sets the following registry values for the database service. The identifying GUID must be unique for each vendor database of a certain format. For instance, in this code example from the sample, change 6E9D4C5A-55B4-4c52-90B7-DDDC75CA4D50 to your own unique GUID in your INF file.

[DatabaseAddReg]
HKLM,System\CurrentControlSet\Services\WbioSrvc\Databases\{6E9D4C5A-55B4-4c52-90B7-DDDC75CA4D50},BiometricType,0x00010001,0x00000008
HKLM,System\CurrentControlSet\Services\WbioSrvc\Databases\{6E9D4C5A-55B4-4c52-90B7-DDDC75CA4D50},Attributes,0x00010001,0x00000001
HKLM,System\CurrentControlSet\Services\WbioSrvc\Databases\{6E9D4C5A-55B4-4c52-90B7-DDDC75CA4D50},Format,,"00000000-0000-0000-0000-000000000000"
HKLM,System\CurrentControlSet\Services\WbioSrvc\Databases\{6E9D4C5A-55B4-4c52-90B7-DDDC75CA4D50},InitialSize,0x00010001,0x00000020
HKLM,System\CurrentControlSet\Services\WbioSrvc\Databases\{6E9D4C5A-55B4-4c52-90B7-DDDC75CA4D50},AutoCreate,0x00010001,0x00000001
HKLM,System\CurrentControlSet\Services\WbioSrvc\Databases\{6E9D4C5A-55B4-4c52-90B7-DDDC75CA4D50},AutoName,0x00010001,0x00000001
HKLM,System\CurrentControlSet\Services\WbioSrvc\Databases\{6E9D4C5A-55B4-4c52-90B7-DDDC75CA4D50},FilePath,,""
HKLM,System\CurrentControlSet\Services\WbioSrvc\Databases\{6E9D4C5A-55B4-4c52-90B7-DDDC75CA4D50},ConnectionString,,""

For information about INX files and how they differ from INF files, see Using INX Files to Create INF Files.

9. Creating a Device Interface for a WBDI Driver

After the device callback object has been initialized and returned to the driver, at the time of queue setup, the driver should create a device interface instance for the biometric device.

Specifically, WBDI drivers must expose the GUID_DEVINTERFACE_BIOMETRIC_READER device interface by calling IWDFDevice::CreateDeviceInterface:

hr = m_FxDevice->CreateDeviceInterface(&GUID_DEVINTERFACE_BIOMETRIC_READER, NULL);

This call is followed by a call to IWDFDevice::AssignDeviceInterfaceState:

hr = m_FxDevice->AssignDeviceInterfaceState(&GUID_DEVINTERFACE_BIOMETRIC_READER,NULL,TRUE);

A WBDI driver that wants to expose functionality to a legacy (non-WBDI) biometric stack should expose another device interface for legacy applications and make sure that the Exclusive value is set to zero in the INX file that installs the legacy stack.

Exposing the GUID_DEVINTERFACE_BIOMETRIC_READER device interface causes the WBF service to enumerate the driver only. If Exclusive mode is not set, WBF does not attempt to open and control the device.

Alternatively, the driver could detect internally that it is in legacy mode and then not expose the GUID_DEVINTERFACE_BIOMETRIC_READER device interface.

Fingerprint driver development in Windows(2)

David Dong — Wed, 10 Jun 2020 20:57:55 +0800

Continue…

3. Biometric Unit

Biometric devices are manufactured in a wide variety of types and configurations. The Windows Biometric Framework (WBF) addresses this variety by providing an extensible architecture that enables third-party developers to create custom plug-in components. A primary component is a software object called a biometric unit. Biometric units expose the capabilities of a device to the framework through a standard interface that consists of sensor, engine, and storage adapters

A single physical sensor is associated with each biometric unit. The sensor captures biometric data and may also, depending on its hardware capabilities, perform other biometric operations such as template matching and storage. Sensors that do not support onboard matching or storage require additional software modules to perform these functions.

The above diagram shows how data flows through a biometric unit. Essentially, the data flow forms a type of pipeline. The initial input to the pipeline is a biometric sample captured by a physical sensor. The engine attempts to match the sample to templates that already exist in the template store or, if no match is found, builds a new template from repeated physical samples and saves the template in the store.

4. Biometric Unit Life Cycle

4.1 Creation

When the Windows Biometric Service starts or when it receives a hardware notification from the Plug and Play manager, it scans for any hardware that supports the well-known interface

GUID_DEVINTERFACE_BIOMETRIC_READER (E2B5183A-99EA-4cc3-AD6B-80CA8D715B80)

For each biometric device discovered it then:

Opens a handle to the device.
Queries the device capabilities by using a Windows Biometric Driver control interface.
Opens the device registry key and attempts to locate information about any associated adapter plug-ins.
Opens the Windows Biometric Service registry key and searches for information about any global plug-ins that should override the device-specific values found in the preceding step.
Builds a biometric unit to represent the device and passes the unit to a biometric service provider.

4.2 Configuration

After a biometric service provider (BSP) accepts a biometric unit, it configures the unit and assigns it to a sensor pool. The unit is configured by loading the appropriate adapters, connecting to a template store, and possibly changing the operating mode. A biometric unit can be configured to operate in one of two modes:

In the basic mode, the unit acts as a simple capture device. It does not use onboard processing or storage but relies entirely on software adapters for additional support. It should be able to generate samples in a standard format. For example, fingerprint readers should generate samples that comply with ANSI INCITS 381-2004. The purpose of the basic mode is to support interoperability among components from different vendors.
In the advanced mode, the biometric unit uses onboard processing and storage functions. It must expose the standard interfaces but can generate and process samples in any format desired.

4.3 Shut Down

When the Windows Biometric Service shuts down or when the Plug and Play manager notifies it that a unit has been removed, the service deletes all biometric units.

5. Adapters

Plug-in software components called adapters connect a biometric unit to its underlying hardware and supply any functionality that may be missing from the sensor hardware. There are three types of adapters that you can create:

A sensor adapter wraps a biometric device and provides a standard interface for configuring the sensor, capturing samples, and controlling the flow of biometric data to the processing engine.
An engine adapter generates biometric templates from captured samples, matches samples to existing templates, and indexes templates.
A storage adapter manages template databases. Adapters can be loaded and unloaded at runtime. This enables the Windows Biometric Service to dynamically reconfigure a biometric unit by connecting it to a different set of adapters.

Finally, each of the sensor, engine and storage adapter interfaces expose two methods, ControlUnit and ControlUnitPrivileged, that enable client applications to access custom adapter capabilities defined by the vendor. This allows the vendor to define an almost unlimited set of control operations for a device. Further, by choosing which function to implement, a vendor can choose to make some control operations available to non-privileged users while restricting other operations to privileged users.

5.1 Sensor Adapter Function

A sensor adapter wraps a biometric device and provides a standard interface that enables the Windows Biometric service to configure the sensor, capture samples, and control the flow of biometric data to the processing engine. The following functions must be implemented by the developer of the adapter. They are called by the Windows Biometric service.

Interface	Description
SensorAdapterAcceptCalibrationData	Passes calibration data from the engine adapter to the sensor adapter.
SensorAdapterActivate	Gives the Sensor Adapter the chance to perform any work needed to bring the sensor component out of an idle state.
SensorAdapterAttach	Adds a sensor adapter to the processing pipeline of the biometric unit.
SensorAdapterCancel	Cancels all pending sensor operations.
SensorAdapterClearContext	Prepares the processing pipeline of the biometric unit for a new operation.
SensorAdapterControlUnit	Performs a vendor-defined control operation that does not require elevated privilege.
SensorAdapterControlUnitPrivileged	Performs a vendor-defined control operation that requires elevated privilege.
SensorAdapterDeactivate	Gives the Sensor Adapter the chance to perform any work needed to put the sensor component into an idle state.
SensorAdapterDetach	Releases adapter specific resources attached to the pipeline.
SensorAdapterExportSensorData	Retrieves the most recently captured biometric sample formatted as a standard WINBIO_BIR structure.
SensorAdapterFinishCapture	Called by the Windows Biometric Framework to wait for the completion of a capture operation initiated by the SensorAdapterStartCapture function.
SensorAdapterGetIndicatorStatus	Retrieves a value that indicates whether the sensor indicator is on or off.
SensorAdapterNotifyPowerChange	Receives notification about a change in the computer power state and prepares the sensor adapter accordingly.
SensorAdapterPipelineCleanup	Gives the Sensor Adapter the chance to perform any cleanup in that requires help from the Engine or Storage adapter components.
SensorAdapterPipelineInit	Gives the Sensor Adapter the chance to perform any initialization that remains incomplete, and which requires help from the Engine or Storage adapter components.
SensorAdapterPushDataToEngine	Makes the current contents of the sample buffer available to the engine adapter.
SensorAdapterQueryCalibrationFormats	Determines the set of calibration formats supported by the Sensor Adapter.
SensorAdapterQueryExtendedInfo	Determines the capabilities and limitations of the biometric sensor component.
SensorAdapterQueryStatus	Retrieves information about the current status of the sensor device.
SensorAdapterReset	Reinitializes the sensor.
SensorAdapterSetCalibrationFormat	Notifies the sensor adapter that a particular calibration data format has been selected by the engine adapter.
SensorAdapterSetIndicatorStatus	Toggles the sensor indicator on or off.
SensorAdapterSetMode	Sets the sensor adapter mode.
SensorAdapterStartCapture	Begins an asynchronous biometric capture.
WbioQuerySensorInterface	Retrieves a pointer to the WINBIO_SENSOR_INTERFACE structure for the sensor adapter.

5.2 Engine Adapter Function

An engine adapter generates biometric templates from captured samples, matches samples to existing templates, and indexes templates. The following functions must be implemented by the adapter developer. They are called by the Windows Biometric service.

Interface	Description
EngineAdapterCreateKey	Called by the Windows Biometric Framework to push an HMAC key to the sensor. The returned key identifier will be passed back to the biometric unit when the framework calls EngineAdapterIdentifyFeatureSetSecure.
EngineAdapterAcceptSampleData	Accepts a raw biometric sample and extracts a feature set.
EngineAdapterActivate	Gives the Engine Adapter the chance to perform any work needed to bring the sensor component out of an idle state.
EngineAdapterAttach	Adds an engine adapter to the processing pipeline of the biometric unit.
EngineAdapterCheckForDuplicate	Determines whether a new template in the pipeline duplicates any template already saved in the database regardless of the identity associated with the templates.
EngineAdapterClearContext	Prepares the processing pipeline of the biometric unit for a new operation.
EngineAdapterCommitEnrollment	Finalizes the enrollment object, converts it to a template, and saves the template in the database.
EngineAdapterControlUnit	Performs a vendor-defined control operation that does not require elevated privilege.
EngineAdapterControlUnitPrivileged	Performs a vendor-defined control operation that requires elevated privilege.
EngineAdapterCreateEnrollment	Initializes the enrollment object in the biometric unit pipeline.
EngineAdapterDeactivate	Gives the Engine Adapter the chance to perform any work needed to put the sensor component into an idle state.
EngineAdapterDetach	Releases adapter-specific resources attached to the pipeline.
EngineAdapterDiscardEnrollment	Deletes intermediate enrollment state information from the pipeline.
EngineAdapterExportEngineData	Retrieves a copy of the most recently processed feature set or template from the engine in a standard biometric information record.
EngineAdapterGetEnrollmentHash	Retrieves the hash of the completed enrollment template in the pipeline.
EngineAdapterGetEnrollmentStatus	Determines whether the enrollment object is ready to be committed to the pipeline.
EngineAdapterIdentifyAll	Determines the identities of any people who are currently in camera frame.
EngineAdapterIdentifyFeatureSet	Builds a template from the current feature set and locates a matching template in the database.
EngineAdapterIdentifyFeatureSetSecure	Called by the Windows Biometric Framework to build a template from the current feature set and locate a matching template in the database. If a match can be found, the engine adapter must fill the Identity, SubFactor, Authorization, and AuthorizationSize fields.
EngineAdapterNotifyPowerChange	Receives notification about a change in the computer power state and prepares the engine adapter accordingly.
EngineAdapterPipelineCleanup	Gives the Engine Adapter the chance to perform any cleanup that requires help from the Storage Adapter.
EngineAdapterPipelineInit	Gives the Engine Adapter the chance to perform any initialization that remains incomplete.
EngineAdapterQueryCalibrationData	Gets a set of post-capture calibration data from the engine adapter.
EngineAdapterQueryExtendedInfo	Determines the capabilities and limitations of the biometric engine component.
EngineAdapterQueryHashAlgorithms	Retrieves an array of object identifiers that represent the hash algorithms supported by the engine adapter.
EngineAdapterQueryIndexVectorSize	Retrieves the size of the index vector used by the engine adapter.
EngineAdapterQueryPreferredFormat	Determines the input data format preferred by the engine adapter.
EngineAdapterQuerySampleHint	Retrieves the number of correct samples required by the engine adapter to construct an enrollment template.
EngineAdapterRefreshCache	Notifies the Engine Adapter that it should discard any cached templates that it may be keeping in memory.
EngineAdapterSelectCalibrationFormat	Called by the Windows Biometric Framework to determine which of the Sensor Adapter s calibration formats the Engine Adapter wants to use.
EngineAdapterSetAccountPolicy	Sets the extended default and per-user antispoofing policies used by the engine adapter.
EngineAdapterSetEnrollmentParameters	Gives the engine adapter additional information about an enrollment operation.
EngineAdapterSetEnrollmentSelector	Tells the Engine Adapter which person to track for the current enrollment operation.
EngineAdapterSetHashAlgorithm	Selects a hash algorithm for use in subsequent operations.
EngineAdapterUpdateEnrollment	Adds the current feature set to the enrollment object.
EngineAdapterVerifyFeatureSet	Compares the template in the current feature set with a specific template in the database.
WbioQueryEngineInterface	Retrieves a pointer to the WINBIO_ENGINE_INTERFACE structure for the engine adapter.

5.3 Storage Adapter Functions

A storage adapter manages template databases. The following functions must be implemented by the adapter developer. They are called by the Windows Biometric service.

Interface	Description
StorageAdapterActivate	Gives the Storage Adapter the chance to perform any work needed to bring the storage component out of an idle state.
StorageAdapterAddRecord	Adds a template to the database.
StorageAdapterAttach	Adds a storage adapter to the processing pipeline of the biometric unit.
StorageAdapterClearContext	Prepares the processing pipeline of the biometric unit for a new operation.
StorageAdapterCloseDatabase	Closes the database associated with the pipeline and frees all related resources.
StorageAdapterControlUnit	Performs a vendor-defined control operation that does not require elevated privilege.
StorageAdapterControlUnitPrivileged	Performs a vendor-defined control operation that requires elevated privilege.
StorageAdapterCreateDatabase	Creates and configures a new database.
StorageAdapterDeactivate	Gives the Storage Adapter the chance to perform any work needed to put the storage component into an idle state.
StorageAdapterDeleteRecord	Deletes one or more templates from the database.
StorageAdapterDetach	Releases adapter-specific resources attached to the pipeline.
StorageAdapterEraseDatabase	Erases the database and marks it for deletion.
StorageAdapterFirstRecord	Positions the result set cursor on the first record in the set.
StorageAdapterGetCurrentRecord	Retrieves the contents of the current record in the pipeline result set.
StorageAdapterGetDatabaseSize	Retrieves the database size and available space.
StorageAdapterGetDataFormat	Retrieves format and version information used by the current database associated with the pipeline.
StorageAdapterGetRecordCount	Retrieves the number of template records in the pipeline result set.
StorageAdapterNextRecord	Advances the result set cursor by one record.
StorageAdapterNotifyPowerChange	Receives notification about a change in the computer power state and prepares the storage adapter accordingly.
StorageAdapterOpenDatabase	Opens a database for use by the storage adapter.
StorageAdapterPipelineCleanup	Gives the Storage Adapter the chance to perform any cleanup in preparation for closing the template database.
StorageAdapterPipelineInit	Gives the Storage Adapter the chance to perform any initialization that remains incomplete.
StorageAdapterQueryByContent	Queries the database that is currently open for templates associated with a specified index vector.
StorageAdapterQueryBySubject	Queries the database that is currently open for templates associated with a specified identity and sub-factor.
StorageAdapterQueryExtendedInfo	Determines the capabilities and limitations of the biometric storage component.
WbioQueryStorageInterface	Retrieves a pointer to the WINBIO_STORAGE_INTERFACE structure for the storage adapter.

6. IOCTLs Interface of Biometric

WBDI is a Windows standard IOCTL-based interface. When you write a WBDI driver, you must support a set of mandatory IOCTLs. You may additionally choose to support optional IOCTLs.

Biometric winbio_ioctl.h contains the following programming IOCTLs interfaces:

6.1 IOCTLs

Interface	Description
IOCTL_BIOMETRIC_CALIBRATE	The IOCTL_BIOMETRIC_CALIBRATE IOCTL directs the driver to perform any necessary steps to calibrate the device for use.
IOCTL_BIOMETRIC_CAPTURE_DATA	The IOCTL_BIOMETRIC_CAPTURE_DATA IOCTL directs the driver to retrieve the next scan of biometric data. This call should put the device into capture mode.Vendor-supplied WBDI drivers must support IOCTL_BIOMETRIC_CAPTURE_DATA.
IOCTL_BIOMETRIC_GET_ATTRIBUTES	The IOCTL_BIOMETRIC_GET_ATTRIBUTES IOCTL returns a structure that contains a set of attributes for the sensor. Vendor-supplied WBDI drivers must support this IOCTL.
IOCTL_BIOMETRIC_GET_INDICATOR	The IOCTL_BIOMETRIC_GET_INDICATOR IOCTL directs the driver to retrieve the status of the indicator light. This IOCTL is optional.
IOCTL_BIOMETRIC_GET_SENSOR_STATUS	The IOCTL_BIOMETRIC_GET_SENSOR_STATUS IOCTL tells the driver to perform any necessary steps to collect the current operating status of the device. Vendor-supplied WBDI drivers must support this IOCTL.
IOCTL_BIOMETRIC_RESET	The IOCTL_BIOMETRIC_RESET IOCTL resets the device to a known or idle state, according to the current power state. Vendor-supplied WBDI drivers must support this IOCTL.
IOCTL_BIOMETRIC_SET_INDICATOR	The IOCTL_BIOMETRIC_SET_INDICATOR IOCTL directs the driver to update the status of the indicator light.
IOCTL_BIOMETRIC_UPDATE_FIRMWARE	The IOCTL_BIOMETRIC_UPDATE_FIRMWARE IOCTL tells the driver to update the firmware for the device with the given firmware image. This IOCTL is optional.

6.2 Biometric IOCTLs Calling Sequence

A vendor-supplied WBDI driver should be prepared to receive IOCTL requests in the following order from the Windows Biometric Service (WBS). You can review examples of handlers for these IOCTLs in Device.cpp in WudfBioUsbSample.

The Windows Biometric Service or the sensor adapter initializes the biometric device and verifies that it is ready for use. The service or adapter sends an IOCTL_BIOMETRIC_GET_ATTRIBUTES request.
The driver receives a pointer to a WINBIO_SENSOR_ATTRIBUTES structure. In the handler for this IOCTL, the driver should fill in the relevant members of this structure and complete the request by calling IWDFIoRequest::Complete.
Next, the driver receives IOCTL_BIOMETRIC_GET_SENSOR_STATUS. The driver should fill in the relevant members of the WINBIO_DIAGNOSTICS structure and complete the request.
If the driver indicates that calibration is necessary in the SensorStatus member of the WINBIO_DIAGNOSTICS structure that was returned from the IOCTL_BIOMETRIC_GET_SENSOR_STATUS request, the driver next receives an IOCTL_BIOMETRIC_CALIBRATE request. The driver must provide a handler for this IOCTL. After calibrating the device, the callback should return a WINBIO_CALIBRATION_INFO structure.
The driver can now expect to receive IOCTL_BIOMETRIC_CAPTURE_DATA requests. Because only one capture can be pending at any time, the handler for this request should first confirm that no request is pending. If a request is pending, complete the request with WINBIO_E_DATA_COLLECTION_IN_PROGRESS.
The WinBio service or an application can request cancellation of an outstanding capture request at any time by calling Win32 cancellation routines such as CancelIo, CancelIoEx, or CancelSynchronousIo. As such, WBDI drivers must also support cancellation.
The driver handles cancellation by calling IWDFIoRequest::MarkCancelable to register an IRequestCallbackCancel interface.
The handler then programs the device for capture mode and returns from the callback. The request should remain pending until canceled or the driver detects that the capture is complete. After this I/O request is completed, the device can return to an idle state. A client may make an initial call to IOCTL_BIOMETRIC_CAPTURE_DATA to determine the correct buffer size for an actual capture.
The handler for IOCTL_BIOMETRIC_RESET should physically reset the device to a known or idle state. The handler for this request must also cancel any pending data collection I/O and fill out the WINBIO_BLANK_PAYLOAD structure. The handler then completes the request. Clients do not need to call reset between calls to IOCTL_BIOMETRIC_CAPTURE_DATA.

Fingerprint driver development in Windows(1)

David Dong — Sun, 07 Jun 2020 21:19:23 +0800

Windows 7 and later implements support for Biometric devices. The Windows Biometric Framework (WBF) is a generic biometric architecture in Windows 7 and later versions of Windows.

WBF includes an IOCTL-based driver interface known as the Windows Biometric Driver Interface (WBDI) as well as a Windows service called the Windows Biometric Framework API (Windows) (WBS).

WBDI drivers respond to requests from the WBS(WinBio) service. WBF also includes Windows log-in support.

Starting with some concepts.

1. Basic Concepts

WBF:

Windows Biometric Framework. WBF is a generic biometric architecture in Windows 7 and later versions of Windows. WBF includes an IOCTL-based driver interface known as the Windows Biometric Driver Interface (WBDI) as well as a Windows service called the Windows Biometric Service (WBS).

WBDI:

Windows Biometric Driver Interface. WBDI is a programming interface that a biometric driver can use to expose the biometric device through the Windows Biometric Service (WBS). The developer can implement a WBDI driver by using any supported driver technology, including the following.

User Mode Driver Framework (UMDF)
Kernel Mode Driver Framework (KMDF)
Windows Driver Model (WDM)

A WBDI biometric driver must also support the WBDI driver interface GUID and all mandatory I/O controls (IOCTLs).

WBS:

Windows Biometric Service. The Windows Biometric Service manages installed biometric drivers and supports the Windows Biometric Framework API to provide device access to client applications.

Adapters:

Plug-in software components that connect a biometric unit to its underlying hardware and supply any functionality that may be missing from the sensor hardware. There are three types of adapters that you can create:

A sensor adapter wraps a biometric device and provides a standard interface for configuring the sensor, capturing samples, and controlling the flow of biometric data to the processing engine.
An engine adapter generates biometric templates from captured samples, matches samples to existing templates, and indexes templates.
A storage adapter manages template databases.

WDF:

Windows Driver Foundation.

WUDF:

Windows User-Mode Driver Framework. Also referred to as UMDF.

UMDF:

User-Mode Driver Framework.

KMDF:

Kernel-Mode Driver Framework.

Wudfhost(exe):

Windows User-mode Driver Framework Driver host process. The driver host process loads the DLL of the UMDF driver and framework provided by the vendor and provides the execution environment for routing messages between the driver in the user-mode driver and the driver in the user-mode stack.

WDM:

Windows Driver Manager. The driver manager is a Windows service that manages all instances of the wudfhost driver host process. The driver manager starts and tracks information about each driver host process. Each host is a child of the driver manager.

Biometric Unit:

The central component of the Windows Biometric Framework plug-in architecture, a software object that exposes the capabilities of a biometric device to the framework through a standard interface.

Overview

The following figure shows how the UMDF based windows biometric driver interface (WBDI) driver can be used for windows biometric framework (WBF) biometric support in Windows 7. All biometric operations rely on client applications to windows biometric services (WBS). The WBS sends the request to the biometric device driver that exposes the WBDI interface.

2. UMDF

UMDF is a Framework for creating user-mode drivers. Like the kernel-mode driver framework (KMDF), UMDF provides an abstraction layer from WDM that handles a large number of plug and play (PNP) and power management functions and allows the driver to choose to enable specific functions and event handling. UMDF driver abstracts hardware functions, runs in user mode environment, and can access various services.

The UMDF driver interacts with the following components provided by the system.

Wudfhost(exe)
WDM
Reflector

2.1 UMDF advantages

Because UMDF driver running on the user space, which results in the stability of operating systems because they only have access to the address space in which they run the process.
Because the local service account runs under the UMDF driver, it has limited access to user data or system files.
User-mode drivers run in a simpler environment than KMDF.
UMDF-V2 provides most of the functions of KMDF.
UMDF version 2 facilitates conversion between KMDF and UMDF.

2.2 When to use UMDF?

On most occasions, Microsoft recommends writing UMDF driver instead of KMDF, since the two frameworks share multiple interfaces, you can also later convert the driver to KMDF if necessary.
However, the following features are only available for KMDF drivers. If the driver requires one of these features, you must write a KMDF driver.

DMA
Bus Enumeration
Power control (only limited support in UMDF)
Access WDM object and lrp
non-cache I/O access
WMI
IOCTLs

2.3 UDMF architecture

The above diagram shows how the driver manager builds a user-mode device stack, and how the host process, reflector, and driver manager to process an I/O request that an application sends to a User-Mode Driver Framework (UMDF) driver.

Similar to a kernel-mode stack, the construction and tear down of a user-mode stack is driven by Plug and Play (PnP) events. After the kernel-mode stack has been built, the reflector notifies the driver manager to start the construction of the user-mode stack. The driver manager launches the driver host process and provides sufficient information to the launched process to build the user-mode stack.

The driver host process provides the execution environment for user-mode drivers and routes messages between drivers in the user-mode stack. The reflector uses a message-based interprocess communication mechanism to communicate with the driver manager and host process.

To send an I/O request to a UMDF driver, an application calls a Win32 file I/O function, such as CreateFile, ReadFileEx, CancelIoEx, or DeviceIoControl. When the reflector receives a request from the client application, it sends the request to the appropriate driver host process. The driver host process then routes the request to the top of the correct user-mode device stack.

The request is either completed by one of the drivers in the user-mode stack or forwarded by one of the drivers back to the reflector. When the reflector receives a request from the user-mode driver stack, it sends the request down the kernel-mode stack for completion.

If no special requirements, the fingerprint driver normally uses the UMDF model to develop.

Use TraceView to trace windows driver log

David Dong — Thu, 04 Jun 2020 20:14:41 +0800

For the windows driver development as I am doing these days, Microsoft provides a log printing and log viewing mechanism, which can view the logs of specified driver files, filter the logs according to LEVEL and FLAG, or save them as files. This mechanism is WPP and TraceView.exe.

How to use?

The usage is simple, the WDF framework has built in the function and enclosed the WPP into the trace.h. The driver program imports the header file and adds TraceEvents() method to the place where needs to print log. Use the TraceView.exe application to capture and view the logs when the driver program running.

TraceEvents(TRACE_LEVEL_INFORMATION, DBG_INIT, "log_name");

The first two parameters of this method represent the LEVEL and FLAG which will be used to control the logs output level and type.

In Windows10, the TraceView.exe is putted into the folder at WDK install path, normally the default installation path is

C:\Program Files (x86)\Windows Kits\10\Tools\x64

Run the TraceView.exe application and create a new log section.

Click Add Provider and select the PDB file of your driver program.

The trace provider will generate trace event messages.

Click Next button and choose Set Flag and Level, in the pop window right click Level and select the print level of the log. Suggest selecting verbose to print the logs as much as possible in the debug phase.

Click ‘OK’ button to complete the configuration.

Start running the driver program and the TraceView will output the logs printed by TraceEvents() in the code.

Recommend some articles for the WPP/TraceView introduction. https://blog.csdn.net/xiangbaohui/article/details/106424665
https://blog.csdn.net/u012308586/article/details/94429941
https://docs.microsoft.com/zh-cn/windows-hardware/drivers/devtest/enabling-wpp-tracing-through-windows-event-log

HelloWorld: my first SGX application

David Dong — Mon, 01 Jun 2020 22:04:51 +0800

A simple sample code to get started with the SGX application development.

New create VC++ Win32 Console Application. The project name is HelloWorld.
Create new VC++ Intel SGX Enclave project. Use default project name Enclave1.
Edit Enclave1.edl file with below code. This piece of code declares the foo() method as a trusted method and executes at trusted zone.

enclave {
    from "sgx_tstdc.edl" import *;

    trusted {
        /* define ECALLs here. */
		public void foo([out,size=len] char* buf,size_t len);
    };

    untrusted {
        /* define OCALLs here. */

    };
};

Edit Enclave1.cpp. Realize the foo() method.

#include "sgx_trts.h"
#include "Enclave1_t.h"
#include "sgx_trts.h"
#include <string.h>
void foo(char *buf, size_t len)
{
	const char *secret = "Hello App!";
	if (len > strlen(secret))
	{
		memcpy(buf, secret, strlen(secret) + 1);
	}
}

Set the Enclave1 project configuration as below.
Build the Enclave1 project. Above result shows the Enclave1 project has been build successfully. Next I need to add it into the HelloWorld project and call the foo() methods.
Edit the main() method of HelloWorld.cpp file.

#include "stdafx.h"
#include <stdio.h>
#include <tchar.h>
#include "sgx_urts.h"
#include <string.h>
#include "Enclave1_u.h"
#define ENCLAVE_FILE _T("Enclave1.signed.dll")
#define MAX_BUF_LEN 100
int main()
{
	sgx_enclave_id_t	eid;
	sgx_status_t		ret = SGX_SUCCESS;
	sgx_launch_token_t	token = { 0 };
	int updated = 0;
	char buffer[MAX_BUF_LEN] = "Hello World!";
	//create a enclave container
	ret = sgx_create_enclave(ENCLAVE_FILE, SGX_DEBUG_FLAG, &token, 
    &updated, &eid, NULL);
	if (ret != SGX_SUCCESS)
	{
		printf("APP:error %#x ,failed to create enclave .\n", ret);
		return -1;
	}
	//Enclave CALL(ECALL) 
	foo(eid, buffer, MAX_BUF_LEN);
	printf("%s\n", buffer);
	getchar();
	//distory enclave container
	if (SGX_SUCCESS != sgx_destroy_enclave(eid))
		return -1;
	system("pause");
	getchar();
	return 0;
}

Set the HelloWorld project configuration.
Add the Enclave1 project into the HelloWorld project. Right click ‘Solution HelloWorld’ -> add -> existing project and select Enclave1 project. Now there are two projects under the ‘Solution HelloWorld’. right click HelloWorld project -> Intel SGX Configuration -> Import Enclave Select Enclave1.edl.
The Enclave1.edl file will be envoloved into the source of HelloWorld project.
Since the HelloWorld project is the main project, I need to add the dependency of Enclave1 project. Set the main project. Set the dependency.
The configuration is done!
Build the main project and start to run.
Unfortunately, the trying is failed with the below error. That is because my CPU cannot support SGX at present. I have to change the running mode to Simulation. I got below result, which means my first app was runing well.:v:

Configure Intel SGX on Win10

David Dong — Sun, 31 May 2020 22:42:44 +0800

The first step we should know if the hardware support SGX.
Which include:

BIOS support?
CPU support?
Software packages installed?

Now let us go through the above one by one.

CPU Query

Get the CPU model of your hardware. By device manager -> processors
Find the corresponding model on the Intel official website, which will indicate whether it supports SGX.
Oh! My laptop’s CPU supports but need install IME software.
Ok, remember this requirement and do it later. 😎

BIOS Query

There is 3 kind of settings on the system BIOS for SGX.

Enable
Software controlled - enabled through software applications. If Intel SGX is set to software-controlled, Intel SGX is initially disabled. You need to make the following calls in the SDK through software application to set it to enabled state:

sgx_enable_device()
sgx_cap_enable_device()

Set software control mode is helpful to reduce consumption of system resources otherwise the SGX is always on and occupies a larger amount of RAMs which affects other programs and processes.

Disable

Support software package

Intel SGX software package list Download SGX software packages at here.
Install IME (Intel Management Engine) Download IME package at here. Run SetupME.exe
Microsoft Visual Studio 2015/2017
Should install Microsoft Visual Studio 2015/2017 before install SGX SDK/PSW, it is important here for the installation sequence. The SGX SDK will install the plug-in on Visual Studio, if SGX SDK is installed firstly, Visual Studio will lose the plug-in.
The Visual Studio install package is at here.
Install SGX SDK

Run Intel(R)_SGX_Windows_SDK_2.7.101.2.exe.
Install PSW
Run Intel SGX PSW for Windows v2.7.101.2.exe and unzip.
You will get a folder Check the build number of operating system by winver command.
- If the system is windows 10 fall creators update (version 1709) or later, enter PSW_INF_RS3_and_above folder, run windows PowerShell as administrator to open command line window, and then enter PSW_INF_RS3_and_above device directory, keep going down until find sgx_base.inf file, input the following command:

pnputil /add-driver sgx_base.inf  /install

Go back to PSW_INF_RS3_and_above component directory, go down until find sgx_psw.inf file, input the following command:

pnputil /add-driver sgx_psw.inf  /install

If it is the previous version, choose PSW_EXE_RS2_and_before folder and run
Intel(R)_SGX_Windows_x64_PSW_2.7.101.2.exe to install.
Open the Visual Studio 2015 and create a new VC++ project, if the Intel SGX Enclave project can be found in the template window, it means the SDK and Visual Studio plug-in has been installed successfully.

Some notes for WSL

David Dong — Mon, 25 May 2020 22:53:27 +0800

Recently I’m using WSL(Window Subsystem for Linux) for program development. This article presents some short notes for the WSL installation and usage.

What is WSL?

Below is the official introduction of WSL, I copied it here as a startup.

The Windows Subsystem for Linux lets developers run a GNU/Linux environment – including most command-line tools, utilities, and applications – directly on Windows, unmodified, without the overhead of a virtual machine.
You can:

Choose your favorite GNU/Linux distributions from the Microsoft Store.

Run common command-line tools such as grep, sed, awk, or other ELF-64 binaries.

Run Bash shell scripts and GNU/Linux command-line applications including:

Tools: vim, emacs, tmux.

Languages: NodeJS, Javascript, Python, Ruby, C/C++, C# & F#, Rust, Go, etc.

Services: SSHD, MySQL, Apache, Lighttpd, MongoDB, PostgreSQL.

Install additional software using own GNU/Linux distribution package manager.

Invoke Windows applications using a Unix-like command-line shell.

Invoke GNU/Linux applications on Windows.

How to install WSL?

You can refer to installation guidance, the introduction is detailed. You can complete the installation of WSL1 and WSL2 easily just by following the guidance step by step.

How to use?

On the above procedure, you will be requested to choose one Linux OS to install, the choice has

Ubuntu 16.04 LTS
Ubuntu 18.04 LTS
openSUSE Leap 15.1
SUSE Linux Enterprise Server 12 SP5
SUSE Linux Enterprise Server 15 SP1
Kali Linux
Debian GNU/Linux
Fedora Remix for WSL
Pengwin
Pengwin Enterprise
Alpine WSL
After installing the Linux OS at the Windows store.

I choose Ubuntu 16.04 LTS as my Linux system and installed it. The system installation is quite faster than installing on a virtual machine from my own experience. During the process, no need to input any information. After installation, just open the CMD console and input ubuntu, you will run Ubuntu on your Windows system.

Here are some useful tips.

use wslconfig command

In the CMD console, input below command to set the default Linux system if you have multiple.

wslconfig /setdefault <DistributionName>

use below command to uninstall the specified Linux system.

wslconfig /unregister <DistributionName>

use below command to check the available system on you computer.

wslconfig /list

The output of my computer is

C:\WINDOWS\System32>wslconfig /list
Windows Subsystem for Linux Distributions:
Ubuntu-16.04 (Default)

set the default user

If you want to change the user, you can use the below command For example, you want to set the default user as root on the Ubuntu.

ubuntu config --default-user root

WSL file system and local file system access

Access local files in the WSL system. It is easy to access the local file in the WSL system, the local file system is mounted at the /mnt folder. There are c, d, e, and other folders in the /mnt directory, which respectively represent the local C disk, D disk, and E disk. You can directly cd them to the corresponding path.
Access WSL file in the local system. It is not recommended to do so, at least do not try to modify any file externally. Otherwise, it might cause some problems.

Actually, the root directory of WSL can be found at local system path.

C:\Users\XXXX\AppData\Local\Packages
\CanonicalGroupLimited.UbuntuonWindows_79rhkp1fndgsc\LocalState\rootfs

My album is online now!

David Dong — Thu, 23 Apr 2020 21:23:16 +0800

My album is online now! Welcome to visit My Album.

This website collected some interesting stories and photos of my travel experiences. I love to explore all kinds of landscapes and experience different custom around the world, which always make me feel the beauty of nature and bring endless power to me. I hope those posts in my album will bring the same feeling to you, I hope you like it. 🙂

Run virtual machine automatically at windows10 startup

David Dong — Sat, 04 Apr 2020 10:45:40 +0800

I prefer working under Ubuntu because it is a programmer-friendly operating system and helps me develop more efficiently. However, there are times when I need to use Windows because it has more software options available. To address this, I used to install a virtual machine on my Windows 10 computer. But, there is a downside to this method. I had to manually start the virtual machine every time, which was a bit inconvenient. Therefore, I found a way to make the virtual machine start automatically when my Windows 10 boots up.

Step 1

I used VMware workstation, to set up boot auto-start using VMware Workstation, we need to understand the command line tool called “vmrun” provided by the software. “vmrun” is a command line tool that starts a virtual machine by command.

vmrun [AUTHENTICATION-FLAGS] COMMAND [PARAMETERS]

For example, starting a virtual machine with Workstation on a Windows host

vmrun -T ws start "c:\my_VMs_folder\myVM.vmx"

here the -T ws means use VMware Workstation

-T <hostType> (ws|fusion||player)

so the firsty step is to add the vmrun path in the the windows envrionments variable.

It is easy !

Step 2

Now I have added the vmrun into the Windows environments since I have verified by inputting vmrun command in the CMD and received the output of the command.

C:\Users\HP>vmrun
vmrun version 1.17.0 build-18811642
Usage: vmrun [AUTHENTICATION-FLAGS] COMMAND [PARAMETERS]
...

Next, I will need to write a batch script to start my Ubuntu by the vmrun command. The script is simple just as below.

@echo off
vmrun -T ws start "D:\Virtual_Machine\VM_Ubuntu20.04\UbuntuVM20.04.vmx"

I saved the script as vm-autorun.bat.

Step 3

I need to make the script auto-run when Windows 10 starts. In Windows, adding an autorun task is not a complicated work. Just put the script into the Windows startup directory. So I pressed win+r and input shell:startup. This operation helped me to navigate to the Windows startup directory, after that I copied the vm-autorun.bat to this directory. I rebooted the windows10 and after waiting some seconds, the virtual machine started to run and my Ubuntu was booting! It was working!

Step 4

But there was a small problem which made the solution not perfect. That was during the startup there was a command prompt window (CMD) displayed and it was a little annoying. So I need a bitter more optimized. I used a small tweak to hide the command prompt window, making it invisible. I used the VBScript to execute the batch file. This approach allows me to hide the command prompt window. I replaced the vm-autorun.bat file with the vm-autorun.vbs and wrote the below command into the vbs file.

Set ws = CreateObject("Wscript.Shell")
ws.run "cmd /c D:\Virtual_Machine\vm-autorun.bat", vbhide

In the command, D:\Virtual_Machine\vm-autorun.bat was my bat file path and vbhide meant hide mode - do not show command prompt window.

Save the file and reboot.

It was perfect this time, executed just as I expected.

Use Travis CI to build and deploy project automatically on Github

David Dong — Wed, 01 Apr 2020 13:02:34 +0800

This article will introduce how to use Travis CI to build, test and deploy the Github project automatically. I will take my blog project on Github as an example to explain.

Why do I use Travis CI ?

If you’ve looked at my previous article, you may know my blog was deployed on Github Pages and was built by Jekyll static site generator.

Github Pages are a great approach to build websites. Using a Github repository and with the Jekyll static site generator, we can build static websites easily. It is suitable for building a personal blog.

Although Github Pages can automatically generate a website from a repository containing a Jekyll project, it has some limitations.

To deploy the blog on Github Pages, you just need to commit your website's source code to the Github Repo and don't pay any effort on the building and deployment.
Github Pages will do it for you!

One of them is

we can’t use Jekyll plugins

I need to use plugins to extend the functionality of my website, e.g. for implementation of toc and markdown function enrichment.

Understandably,Github Pages doesn’t allow the plugin for security reasons.

Github Pages Generator uses the Jekyll --safe flag.

The workaround is to generate the site locally and then to push the generated HTML to Github (I’ve interpreted how to do in my previous article). However, I usually use different branches for source code and output static HTML. The dev branch accommodates the source code and the gh-pages branch, which is as a window of website visiting, stores the static website files.

So my current workflow is

modify the source code at the dev branch and use Jekyll build locally.
commit the change to the dev branch.
copy the output static website files which are in the _site folder to \tmp folder.
switch to the local gh-pages branch.
add copied static website file in _site folder to the gh_pages branch.
git add & commit.
push to the remote gh-pages branch.

or I can push the files in _site to the remote gh-pages branch forcibly from the dev branch although it is not an ideal way of routine operation.

You see it! ☹️ Really tedious, isn’t it?

I think it is a poor efficiency and should be improved.

The main motivation for me is to be able to get a tool that helps me do the above works automatically. What I want to do is just committing the code to the remote dev branch (this is the necessary work) and others leave to the tool.

The basic idea is to use the Travis CI continuous integration (CI) service. The function of Travis CI service fully meets my expectation. And the most important point is that it is FREE for open-source projects in Github.

Prerequisites

To start using Travis CI, make sure you have:

Github account
Adding authentication methods to your Github account. You can do it by creating a personal access token (PATs) or connecting to GitHub with SSH Key. If you don’t know how to do, please refer to
- Creating a personal access token (PATs)
- Connecting to GitHub with SSH
Travis CI account and sign up, you can use your Github account ID to log in directly.
Authorize the Travis CI (you will be requested to do it when you sign in to Travis CI for the first time), active your Github repositories that you want to use Travis CI service. Please refer to
- To get started with Travis CI using GitHub
One local git repository for your project.

Workflow

The workflow of Travis CI building and deployment is as follows:

git push to the Github repo triggers Travis CI
Travis CI starts up a virtual machine and installs all required software (mostly Ruby gems)
I use a custom rake task to tell Travis CI how to build the Jekyll site and push the updated content back to Github
Travis CI clones a different branch (either dev or master, depending on the kind of Github repo) that holds the website’s source code.
Travis CI runs Jekyll build with the destination in the other branch (gh-pages)
Travis CI does a git push of the other branch (gh-pages)
Github Pages starts serving the updates site

Depending on the required software that needs to be installed, the whole process takes anywhere between 1 and 5 min and is fully automated.

.travis.yml

Travis CI uses the .travis.yml file to describes the build process. Create a .travis.yml into your project’s repo and edit it.

Don’t forget to commit it and push it to the remote repo once you finished editing!

Lifecycle

The whole job’s life cycle can be divided into several phases.

The main phases are:

install - install any dependencies required
script - run the build script

You can complete basic work with the above minimal life cycle. Besides, Travis CI affords some phases that allow users to insert custom commands.

To get to know the details and learn how to write .travis.yml file, please read

Travis CI Job Lifecycle

Sensitive Data

Travis CI will inevitably use the sensitive data (for example, SSH private key or Github personal access token) to access the Github. Of course, we cannot expose the data without any protection. Travis CI provides encryption scheme to protect your secret information.

Travis CI offers commands that can encrypt the private key or any files. You can also save your sensitive data to environment variables, where the data is invisible and can be accessed from any stage in your build process.

For my project, I’ve tried two methods as below:

use SSH scheme access Github with RSA key pairs
access with encrypted Github PATs (personal access tokens)

SSH

For SSH access, I assume you have created SSH key pairs and it is workable for your Github. If you don’t have, do it by following

Generating a new SSH key and adding it to the ssh-agent

you can test the connection status by inputting the command.

ssh -T git@github.com

If you get a message like this, you are accessing Github successfully.

"Hi [secure]/your repo name ! You have successfully authenticated, but GitHub does not provide shell access."

Then use the below command that Travis CI provides to encrypt the private key.

# Sign in travis-ci.com
travis login --com  --github-token "your github personal access keys (*PATs*)" 
# encrypt the SSH private key
travis encrypt-file "your private key" --add

Above commands sign in Travis CI (for applying the APIs), encrypt the private key, and generate a code snippet (as below) into your .travis.yml file to decrypt the private key during the building process.

openssl aes-256-cbc -K $encrypted_70fbe34e406c_key -iv $encrypted_70fbe34e406c_iv -in blog_id_rsa.enc -out blog_id_rsa -d

then add your SSH private key to the ssh-agent.

# start the ssh-agent in the background
eval `ssh-agent -s`
ssh-add blog_id_rsa

After generated the encrypted key leave it into your code repo.

Do not forget to remove your PRIVATE KEY before you commit code to Github Repo!
Otherwise, you may potentially disclose your private key.

PATs

If using PATs, you should follow below steps:

acquire a PAT, refer to
- Creating a personal access token (PATs)
remember to keep this token’s value carefully since it is only visible for the first time generated.
encrypt the PAT by inputting the command

travis encrypt -r <user name>/<repo name> GH_TOKEN= "value of PATs"

This will output a string looking something like this:

secure: ".... encrypted data ...."

Now you can place it in the .travis.yml file.

You can also skip the above, and add it automatically by running:

travis encrypt -r <user name>/<repo name> GH_TOKEN=PAT --add

GH_TOKEN is the environment variable, which can be accessed in the .travis.yml file.

GH_REF environment variable can be set to “github.com/user-name/repo-name.git”.

Please note that the name of the environment variable and its value are both encoded in the string produced by “travis encrypt.” You must add the entry to your .travis.yml with key secure (underneath the env key). This makes the environment variable GH_TOKEN with the value of PATs available to your program.

Once you got the personal access token and encrypted, you can access the Github repo by the below method:

https://${GH_TOKEN}@${GH_REF}" travis:gh-pages

With this method, you can avoid being request passphrase when accessing.

Example

I attached my blog project’s .travis.yml file here as an example.

language: ruby
rvm:
- 2.6.6
before_install:
# decrypt private key
- openssl aes-256-cbc -K $encrypted_70fbe34e406c_key -iv $encrypted_70fbe34e406c_iv -in blog_id_rsa.enc -out blog_id_rsa -d
# add permission 
- chmod 600 blog_id_rsa

# kick off ssh-agent
- eval "$(ssh-agent -s)"
- ssh-add blog_id_rsa
# remove
- rm blog_id_rsa
install: gem install jekyll

script:
- bundle install
- bundle exec jekyll build

#deploy:
#  provider: script
#  skip_cleanup: true
#  script: bash scripts/deploy.sh
#  on:
#    branch: dev
after_success:
# check the connection to host
- ssh -T git@github.com

# add user account
- git config user.name "${USER}"
- git config user.email "${EMAIL}"

# copy _site to tmp
- cp -r /home/travis/build/${USER}/daviddong.github.io/_site  /tmp/
# create new branch
- git checkout -b travis
# clean 
- rm -rf *
# copy _site 
- cp -r /tmp/_site/* ./
- ls -la
# git add & commit
- git add -A
- git commit -m "travis_ci update gh-pages dev -> 3457aec"
# push remote 
- git push --force "https://${GH_TOKEN}@${GH_REF}" travis:gh-pages

# branch whitelist, 
branches:
  only:
  - master     # monitor master branch
  - dev        # monitor dev branch
addons: ssh_known_hosts:daviddong.github.io
# caching bundler gem packages will speed up b
cache: bundler
# Optional: enable email notifications about the outcome of your builds
notifications:
  email: true
env:
  global:
  - secure: NiBbDRjSP7+KRko5wT0uOfSiS4GqdyLzEBFa/95OdN8mCrEslnAPS2aMWNdm5FBqMh4sotR31KCXtgnvQUk0NhxCEdNLYLvv6lK9MA2oLRhhQaeYMQjeGvCGmZyScew3cLNRdo9rn/9wXVNtZM0YUZ+CTJbpZS+jyC3KfndganTE4PPpkel/BS1BnDt8HGaioeyik805CK3bAmXClYjD/+sPaxAL6dS4BJEZzge16JK3IcJepYY+wCgzWtqhsSmHfqxJ8mFodKh8hNSQRNyOiT7G+kX8GiA7Hl5kZdd4Qia7V8ct/ovOhxL/QkVWQYp7uukjg8ugQgq02SYcMdMs86SodSmdSD6cYez5CCnmpDqfFh8SXOcNyAfqQf7MGmItI9WraTNPWPfkv5pUp/7go/ZnfGNUobnDdZh5B5cVr5xogif+B4UI2vpesZmf6IXH+XntlHlfh2x/DG6hSwywUJEEQLGJjsfiAkdQ2jbQZ9WylbNNFovT0Px2jZaetOQ9cD3ekVCjyPHfW4/vfqx6cfr2kdGjxEiQghfyfQ5bV05gJXKNfDenWFNMI4X+L0ELITjJb+q+l1L8CKF0KLjfh7lJsqWMV61qU7QCM7siU1f7KvUZCKjtNu4Wot4A11O+a5HaCzSjf7zQTuIrRwFb4srLf39rRIqU5agtxzBxAOA=

OK, everything is done here!

Let’s see what happens!

I modify code and push to the remote dev branch, open my Travis CI web page to view the building process.

Yes, my commit event to the dev branch triggers Travis CI to work for the building.

Booting virtual machine …

Setting environment variable…

Continue waiting…, oh! start to decryption and install toolchain…

Installation completed, building now…

Building succeed! The last step, deploy…

It’s done! Looks like the whole process was completed successfully! 👍

Let me go to the Github gh-pages repo to check the result.

GREAT! 😄

The repo was updated successfully with the new contents.

The Travis CI works well, which means I can get back to the normal workflow to develop my blog without extra git operation between different branches.

The workflow is not only valid for blog projects deployed on Github Pages but any open source projects on Github.

The last thing is don’t forget to add a Travis CI logo/link to your README.

Troubleshooting

Building Errors:

Error: iv undefined when executing openssl command.

Possible Reason:
- check the Travis CI website address, it should be https://travis-ci.com rather than https://travis-ci.org.
- run travis login --com instead of travis login --org to log in Travis CI.
- check the configuration file ~/.travis/config.yml, the value of endpoint should be https://api.travis-ci.com/.

repos:
	endpoint: https://api.travis-ci.com/

Error: Being asked for entering passphrase for key /root/.ssh/id_rsa when executing openssl command to do decryption of the SSH private key.

It is because you set the passphrase when generating the SSH key, use ssh-keygen -p command to reset the passphrase to none. Then replace the private key with the new one and re-encrypt the private key.
Error: report the encrypted_xxx file size is incorrect when decryption.

Probably you are using the wrong format encryption file, the Travis CI host is running on Ubuntu OS. So the file format should be UNIX, if you generate an encryption file with DOS format, you will encounter this issue.

The solution is to use dos2unix or fromdos command to convert the format to UNIX.
Error: you got repository not known to https://api.travis-ci.org/: user-name/repo-name when you execute the command

travis encrypt -r 'user-name'/'repo-name' DEPLOY_TOKEN = 'your PATs'

It is because your current repo is not in the repo’s list in the config.yml, add your repo’s name into the config.yml and indicate the endpoint to https://api.travis-ci.com/ manually.

repos:
  gangdong/daviddong.github.io:
    endpoint: https://api.travis-ci.com/
  gangdong/sort-algos:
    endpoint: https://api.travis-ci.com/

Reference

Fingerprint implementation in android 8.0 and later

David Dong — Thu, 09 Jan 2020 20:59:29 +0800

Since Android 8.0, Android has fully introduced the HIDL layer into the framework. The purpose is to separate vendor partition from system partition so that Android is capable to upgrade the framework through OTA without recompiling HAL. Correspondingly, the framework of fingerprint has also been reconstructed. This page will give an introduction about the difference in the fingerprint framework between android 7.0 (and early version) and android 8.0 (and later version).

After the study of the previous three articles,
Android Fingerprint Framework (1)
Android Fingerprint Framework (2)
Android Fingerprint Framework (3)

we have discussed the fingerprint framework on android 7.0 in previous blogs, here give a summary for anyone who has not read these articles yet.

fingerprint framework in Android 7.0

This diagram is the fingerprint framework on the android platform, which I have presented in another article and copied here.

From the top layer, the fingerprint application will start the workflow and this is the fingerprint management entry defined by the Android system layer. In the framework internal, some tasks will be done to handle the request from the application.

init.rc starts up the Fingerprintd process during the system boot-up. Fingerpringd then register IFingerprintDaemon remote service to ServiceManager.
System Server will start fingerprint system service FingerprintService.
SystemServer.java

mSystemServiceManager.startService(FingerprintService.class);

FingerprintService calls the interface of Fingerprintd to communicate with Fingerprint HAL layer.
FingerprintService.java

public IFingerprintDaemon getFingerprintDaemon() {
        if (mDaemon == null) {
            mDaemon = IFingerprintDaemon.Stub.
            asInterface(ServiceManager.getService(FINGERPRINTD));
            if (mDaemon != null) {
                try {
                    mDaemon.asBinder().linkToDeath(this, 0);
                    mDaemon.init(mDaemonCallback);
                    mHalDeviceId = mDaemon.openHal();
                    if (mHalDeviceId != 0) {
                        updateActiveGroup(ActivityManager.getCurrentUser(), null);
                    } else {
                        Slog.w(TAG, "Failed to open Fingerprint HAL!");
                        MetricsLogger.count(mContext,
                        "fingerprintd_openhal_error", 1);
                        mDaemon = null;
                    }
                } catch (RemoteException e) {
                    Slog.e(TAG, "Failed to open fingeprintd HAL", e);
                    mDaemon = null; // try again later!
                }
            } else {
                Slog.w(TAG, "fingerprint service not available");
            }
        }
        return mDaemon;
    }

Fingerprintd calls FingerprintDaemonProxy function to open HAL.
FingerprintDaemonProxy.cpp

int64_t FingerprintDaemonProxy::openHal() {
    ALOG(LOG_VERBOSE, LOG_TAG, "nativeOpenHal()\n");
    int err;
    const hw_module_t *hw_module = NULL;
    if (0 != (err = hw_get_module(FINGERPRINT_HARDWARE_MODULE_ID, &hw_module))) {
        ALOGE("Can't open fingerprint HW Module, error: %d", err);
        return 0;
    }
    if (NULL == hw_module) {
        ALOGE("No valid fingerprint module");
        return 0;
    }

    mModule = reinterpret_cast<const fingerprint_module_t*>(hw_module);

    if (mModule->common.methods->open == NULL) {
        ALOGE("No valid open method");
        return 0;
    }

    hw_device_t *device = NULL;

    if (0 != (err = mModule->common.methods->open(hw_module, NULL, &device))) {
        ALOGE("Can't open fingerprint methods, error: %d", err);
        return 0;
    }

    if (kVersion != device->version) {
        ALOGE("Wrong fp version. Expected %d, got %d", kVersion, device->version);
        // return 0; // FIXME
    }

    mDevice = reinterpret_cast<fingerprint_device_t*>(device);
    err = mDevice->set_notify(mDevice, hal_notify_callback);
    if (err < 0) {
        ALOGE("Failed in call to set_notify(), err=%d", err);
        return 0;
    }

    // Sanity check - remove
    if (mDevice->notify != hal_notify_callback) {
        ALOGE("NOTIFY not set properly: %p != %p", mDevice->notify, 
        hal_notify_callback);
    }

    ALOG(LOG_VERBOSE, LOG_TAG, "fingerprint HAL successfully initialized");
    return reinterpret_cast<int64_t>(mDevice); // This is just a handle
}

The HAL code is at below android path normally.

/hardware/libhardware/include/hardware/fingerprint.h
/hardware/libhardware/modules/fingerprint.c

I drew a flow chart to help understand the whole flow more clearly.

The related source code and android path can be found in the below table. Android 7.0 (NOUGAT)

File	Android Path
init.rc	root/system/core/rootdir/init.rc
fingerprintd.cpp	root/system/core/fingerprintd/fingerprintd.cpp
FingerprintDaemonProxy.h	root/system/core/fingerprintd/
fingerprintdaemonproxy.cpp	root/system/core/fingerprintd/fingerprintdaemonproxy.cpp
SystemServer.java	root/frameworks/base/services/java/com/android/server/SystemServer.java
FingerprintService.java	root/frameworks/base/services/core/ java/com/android/server/fingerprint/FingerprintService.java
hardware.h	root/hardware/libhardware/include/hardware/hardware.h
hardware.c	root/hardware/libhardware/hardware.c

fingerprint framework in Android 8.0

Above is the fingerprint framework of Android 7.0, however in Android 8.0 and later versions, Android has updated the framework and introduced a set of languages called HIDL to define the interface between framework and HAL.

Let’s see the difference.

Android 8.0 adds a sub-directory /interface in the /hardware directory, which includes all HIDL files for the hardware module.

Android 8.0 removed Fingerprintd, instead, FingerprintService accesses HAL by calling HIDL.

We can find the change in getFingerprintDaemon() method.

In Android 7.0
FingerprintService.java

mDaemon = IFingerprintDaemon.Stub.asInterface(ServiceManager.
getService(FINGERPRINTD));

While in Android 8.0, mDaemon is achieved from the service of IBiometricsFingerprint.
FingerprintService.java

mDaemon = IBiometricsFingerprint.getService();

IBiometricsFingerprint is a new fingerprint HIDL interface that was introduced on Android 8.0.
IBiometricsFingerprint.hal use HIDL language format defined a series of standard fingerprint operation interfaces. And biometricsfingerprint.cpp class realized the IBiometricsFingerprint interface.

We may notice that the IBiometricsFingerprint returns a service for caller, actually there is a file in the HIDL sub-directory:
android.hardware.biometrics.fingerprint@2.1-service.rc, which will start fps_hal service.
fingerprint@2.1-service.rc

 service fps_hal /vendor/bin/hw/android.hardware.biometrics.fingerprint@2.1-service
    # "class hal" causes a race condition on some devices due to files created
    # in /data. As a workaround, postpone startup until later in boot once
    # /data is mounted.
    class late_start
    user system
    group system input

The files of the fingerprint HIDL related.

If we look at the Service.cpp, we will find the service actually will create a BiometricsFingerprint instance and register as service.

int main() {
    android::sp<IBiometricsFingerprint> bio = BiometricsFingerprint::getInstance();

    configureRpcThreadpool(1, true /*callerWillJoin*/);

    if (bio != nullptr) {
        bio->registerAsService();
    } else {
        ALOGE("Can't create instance of BiometricsFingerprint, nullptr");
    }

    joinRpcThreadpool();

    return 0; // should never get here
}

In the constructor of BiometricsFingerprint class, it calls openHal() to open HAL module.
BiometricsFingerprint.cpp

BiometricsFingerprint::BiometricsFingerprint() : mClientCallback(nullptr), 
mDevice(nullptr) {
    sInstance = this; // keep track of the most recent instance
    mDevice = openHal();
    if (!mDevice) {
        ALOGE("Can't open HAL module");
    }
}

Let’s check the openHal() function.
BiometricsFingerprint.cpp

fingerprint_device_t* BiometricsFingerprint::openHal() {
    int err;
    const hw_module_t *hw_mdl = nullptr;
    ALOGD("Opening fingerprint hal library...");
    if (0 != (err = hw_get_module(FINGERPRINT_HARDWARE_MODULE_ID, &hw_mdl))) {
        ALOGE("Can't open fingerprint HW Module, error: %d", err);
        return nullptr;
    }

    if (hw_mdl == nullptr) {
        ALOGE("No valid fingerprint module");
        return nullptr;
    }

    fingerprint_module_t const *module =
        reinterpret_cast<const fingerprint_module_t*>(hw_mdl);
    if (module->common.methods->open == nullptr) {
        ALOGE("No valid open method");
        return nullptr;
    }

    hw_device_t *device = nullptr;

    if (0 != (err = module->common.methods->open(hw_mdl, nullptr, &device))) {
        ALOGE("Can't open fingerprint methods, error: %d", err);
        return nullptr;
    }

    if (kVersion != device->version) {
        // enforce version on new devices because of HIDL@2.1 translation layer
        ALOGE("Wrong fp version. Expected %d, got %d", kVersion, device->version);
        return nullptr;
    }

    fingerprint_device_t* fp_device =
        reinterpret_cast<fingerprint_device_t*>(device);

    if (0 != (err =
            fp_device->set_notify(fp_device, BiometricsFingerprint::notify))) {
        ALOGE("Can't register fingerprint module callback, error: %d", err);
        return nullptr;
    }

    return fp_device;
}

Have you found that the function realization is similar to the FingerprintDaemonProxy::openHal()? The native method is called and the HAL module is opened here. After access to the HAL, others are all same under the HAL layer.

So far, we can change the fingerprint framework of Android 8.0 as below.

Compare this flowchart carefully with the last flowchart above, we can find the difference.

The related source code and android path can be found in the below table

File	Android Path
fingerprint@2.1-service	root/hardware/interfaces/biometrics/fingerprint/2.1/default
service.cpp	root/hardware/interfaces/biometrics/fingerprint/2.1/default
BiometricsFingerprint.h	root/hardware/interfaces/biometrics/fingerprint/2.1/default
BiometricsFingerprint.cpp	root/hardware/interfaces/biometrics/fingerprint/2.1/default
IBiometricsFingerprint.hal	root/hardware/interfaces/biometrics/fingerprint/2.1
IBiometricsFingerprintClientCallback.hal	root/hardware/interfaces/biometrics/fingerprint/2.1

Now, I think the main difference of the fingerprint framework on Android 8.0 has been introduced and if you have further questions, you can ask in the comment box, I will reply to you as soon as I can.

Android Fingerprint Framework (3)

David Dong — Sat, 21 Dec 2019 22:24:43 +0800

Following the last two articles, this article will discuss the remaining part of the fingerprint framework on android. This article will end this topic.

In the last article,
Android Fingerprint Framework (2)
We have had an overview of the android fingerprint workflow as below.

Init.rc, starts Fingerprintd and registers the remote service FingerprintDaemon with ServiceManager.
The system loads SystemServer and starts fingerservice.
FingerService gets the object of the remote service FingerprintDaemon, and calls the methods to access the HAL. FingerprintDaemoProxy::openHal() will open the native library xx.So to access hardware.

About HAL

The hardware abstract layer (HAL) of the Android system runs in userspace. It shields the implementation details of the hardware driver module downward and provides hardware access service (JNI or binder) upward. Through the hardware abstraction layer, the Android system is divided into two layers to support hardware devices, one layer is implemented in user space, the other is implemented in kernel space. In a traditional Linux system, the hardware support is completely implemented in kernel space, that is, the hardware support is completely implemented in the hardware driver module.

The hardware abstraction layer of the Android system manages various hardware access interfaces in the form of modules. Each hardware module has a dynamic link library xx.So file. The compilation of these dynamic link libraries needs to conform to certain specifications. In the Android system, each hardware abstraction layer module is described by hw_module_t, and the hardware device is described by hw_device_t.

These definition of these two struct is defined at
hardware.h

android path: root/hardware/libhardware/include/hardware/hardware.h

hardware.h

typedef struct hw_module_t {
    /** tag must be initialized to HARDWARE_MODULE_TAG */
    uint32_t tag;

    /**
     * The API version of the implemented module. The module owner is
     * responsible for updating the version when a module interface has
     * changed.
     *
     * The derived modules such as gralloc and audio own and manage this field.
     * The module user must interpret the version field to decide whether or
     * not to inter-operate with the supplied module implementation.
     * For example, SurfaceFlinger is responsible for making sure that
     * it knows how to manage different versions of the gralloc-module API,
     * and AudioFlinger must know how to do the same for audio-module API.
     *
     * The module API version should include a major and a minor component.
     * For example, version 1.0 could be represented as 0x0100. This format
     * implies that versions 0x0100-0x01ff are all API-compatible.
     *
     * In the future, libhardware will expose a hw_get_module_version()
     * (or equivalent) function that will take minimum/maximum supported
     * versions as arguments and would be able to reject modules with
     * versions outside of the supplied range.
     */
    uint16_t module_api_version;
#define version_major module_api_version
    /**
     * version_major/version_minor defines are supplied here for temporary
     * source code compatibility. They will be removed in the next version.
     * ALL clients must convert to the new version format.
     */

    /**
     * The API version of the HAL module interface. This is meant to
     * version the hw_module_t, hw_module_methods_t, and hw_device_t
     * structures and definitions.
     *
     * The HAL interface owns this field. Module users/implementations
     * must NOT rely on this value for version information.
     *
     * Presently, 0 is the only valid value.
     */
    uint16_t hal_api_version;
#define version_minor hal_api_version

    /** Identifier of module */
    const char *id;

    /** Name of this module */
    const char *name;

    /** Author/owner/implementor of the module */
    const char *author;

    /** Modules methods */
    struct hw_module_methods_t* methods;

    /** module's dso */
    void* dso;

#ifdef __LP64__
    uint64_t reserved[32-7];
#else
    /** padding to 128 bytes, reserved for future use */
    uint32_t reserved[32-7];
#endif

} hw_module_t;

/**
 * Every device data structure must begin with hw_device_t
 * followed by module-specific public methods and attributes.
 */
typedef struct hw_device_t {
    /** tag must be initialized to HARDWARE_DEVICE_TAG */
    uint32_t tag;

    /**
     * Version of the module-specific device API. This value is used by
     * the derived-module user to manage different device implementations.
     *
     * The module user is responsible for checking the module_api_version
     * and device version fields to ensure that the user is capable of
     * communicating with the specific module implementation.
     *
     * One module can support multiple devices with different versions. This
     * Can be useful when a device interface changes in an incompatible way
     * but it is still necessary to support older implementations at the same
     * time. One such example is the Camera 2.0 API.
     *
     * This field is interpreted by the module user and is ignored by the
     * HAL interface itself.
     */
    uint32_t version;

    /** reference to the module this device belongs to */
    struct hw_module_t* module;

    /** padding reserved for future use */
#ifdef __LP64__
    uint64_t reserved[12];
#else
    uint32_t reserved[12];
#endif

    /** Close this device */
    int (*close)(struct hw_device_t* device);

} hw_device_t;

Besides, this header file also declares the module name and two important functions.
hardware.h

/**
 * Name of the hal_module_info
 */
#define HAL_MODULE_INFO_SYM         HMI

/**
 * Get the module info associated with a module by id.
 *
 * @return: 0 == success, <0 == error and *module == NULL
 */
int hw_get_module(const char *id, const struct hw_module_t **module);

/**
 * Get the module info associated with a module instance by class 'class_id'
 * and instance 'inst'.
 *
 * Some module types necessitate multiple instances. For example audio supports
 * multiple concurrent interfaces and thus 'audio' is the module class
 * and 'primary' or 'a2dp' are module interfaces. This implies that the files
 * providing these modules would be named audio.primary.<variant>.so and
 * audio.a2dp.<variant>.so
 *
 * @return: 0 == success, <0 == error and *module == NULL
 */
int hw_get_module_by_class(const char *class_id, const char *inst,
                           const struct hw_module_t **module);

These two functions are realized at
hardware.c

android path: root/hardware/libhardware/hardware.c

From the file, we can find the module search path is as below.
hardware.c

/** Base path of the hal modules */
#if defined(__LP64__)
#define HAL_LIBRARY_PATH1 "/system/lib64/hw"
#define HAL_LIBRARY_PATH2 "/vendor/lib64/hw"
#define HAL_LIBRARY_PATH3 "/odm/lib64/hw"
#else
#define HAL_LIBRARY_PATH1 "/system/lib/hw"
#define HAL_LIBRARY_PATH2 "/vendor/lib/hw"
#define HAL_LIBRARY_PATH3 "/odm/lib/hw"
#endif

static const char *variant_keys[] = {
    "ro.hardware",  /* This goes first so that it can pick up a different
                       file on the emulator. */
    "ro.product.board",
    "ro.board.platform",
    "ro.arch"
};

static const int HAL_VARIANT_KEYS_COUNT =
    (sizeof(variant_keys)/sizeof(variant_keys[0]));

/**
 * Load the file defined by the variant and if successful
 * return the dlopen handle and the hmi.
 * @return 0 = success, !0 = failure.
 */
static int load(const char *id,
        const char *path,
        const struct hw_module_t **pHmi)
{
    int status = -EINVAL;
    void *handle = NULL;
    struct hw_module_t *hmi = NULL;

    /*
     * load the symbols resolving undefined symbols before
     * dlopen returns. Since RTLD_GLOBAL is not or'd in with
     * RTLD_NOW the external symbols will not be global
     */
    handle = dlopen(path, RTLD_NOW);
    if (handle == NULL) {
        char const *err_str = dlerror();
        ALOGE("load: module=%s\n%s", path, err_str?err_str:"unknown");
        status = -EINVAL;
        goto done;
    }

    /* Get the address of the struct hal_module_info. */
    const char *sym = HAL_MODULE_INFO_SYM_AS_STR;
    hmi = (struct hw_module_t *)dlsym(handle, sym);
    if (hmi == NULL) {
        ALOGE("load: couldn't find symbol %s", sym);
        status = -EINVAL;
        goto done;
    }

    /* Check that the id matches */
    if (strcmp(id, hmi->id) != 0) {
        ALOGE("load: id=%s != hmi->id=%s", id, hmi->id);
        status = -EINVAL;
        goto done;
    }

    hmi->dso = handle;

    /* success */
    status = 0;

    done:
    if (status != 0) {
        hmi = NULL;
        if (handle != NULL) {
            dlclose(handle);
            handle = NULL;
        }
    } else {
        ALOGV("loaded HAL id=%s path=%s hmi=%p handle=%p",
                id, path, *pHmi, handle);
    }

    *pHmi = hmi;

    return status;
}

/*
 * Check if a HAL with the given name and surname exists, if so return 0, otherwise
 * otherwise return negative.  On success, the path will contain the path to the HAL.
 */
static int hw_module_exists(char *path, size_t path_len, const char *name,
                            const char *subname)
{
    snprintf(path, path_len, "%s/%s.%s.so",
             HAL_LIBRARY_PATH3, name, subname);
    if (access(path, R_OK) == 0)
        return 0;

    snprintf(path, path_len, "%s/%s.%s.so",
             HAL_LIBRARY_PATH2, name, subname);
    if (access(path, R_OK) == 0)
        return 0;

    snprintf(path, path_len, "%s/%s.%s.so",
             HAL_LIBRARY_PATH1, name, subname);
    if (access(path, R_OK) == 0)
        return 0;

    return -ENOENT;
}

int hw_get_module_by_class(const char *class_id, const char *inst,
                           const struct hw_module_t **module)
{
    int i = 0;
    char prop[PATH_MAX] = {0};
    char path[PATH_MAX] = {0};
    char name[PATH_MAX] = {0};
    char prop_name[PATH_MAX] = {0};


    if (inst)
        snprintf(name, PATH_MAX, "%s.%s", class_id, inst);
    else
        strlcpy(name, class_id, PATH_MAX);

    /*
     * Here we rely on the fact that calling dlopen multiple times on
     * the same .so will simply increment a refcount (and not load
     * a new copy of the library).
     * We also assume that dlopen() is thread-safe.
     */

    /* First try a property specific to the class and possibly instance */
    snprintf(prop_name, sizeof(prop_name), "ro.hardware.%s", name);
    if (property_get(prop_name, prop, NULL) > 0) {
        if (hw_module_exists(path, sizeof(path), name, prop) == 0) {
            goto found;
        }
    }

    /* Loop through the configuration variants looking for a module */
    for (i=0 ; i<HAL_VARIANT_KEYS_COUNT; i++) {
        if (property_get(variant_keys[i], prop, NULL) == 0) {
            continue;
        }
        if (hw_module_exists(path, sizeof(path), name, prop) == 0) {
            goto found;
        }
    }

    /* Nothing found, try the default */
    if (hw_module_exists(path, sizeof(path), name, "default") == 0) {
        goto found;
    }

    return -ENOENT;

found:
    /* load the module, if this fails, we're doomed, and we should not try
     * to load a different variant. */
    return load(class_id, path, module);
}

int hw_get_module(const char *id, const struct hw_module_t **module)
{
    return hw_get_module_by_class(id, NULL, module);
}

hw_get_module() will call the hw_get_module_by_class() function. Firstly, it will read the system property ro.hardware through the property_get() function. If the property is found, it then uses the hw_module_exists() function to check whether the xx.So library exists. If it exists, load it directly else if it does not exist, continue to search for the variant_keys array. Checking system attribute values. If found, load it directly. If it does not exist still, load the default.

Let’s turn back to the FingerprintDaemonProxy::openHal() to see how it call the hw_get_module() function.
FingerprintDaemonProxy.cpp

int64_t FingerprintDaemonProxy::openHal() {
    ALOG(LOG_VERBOSE, LOG_TAG, "nativeOpenHal()\n");
    int err;
    const hw_module_t *hw_module = NULL;
    if (0 != (err = hw_get_module(FINGERPRINT_HARDWARE_MODULE_ID, &hw_module))) {
        ALOGE("Can't open fingerprint HW Module, error: %d", err);
        return 0;
    }
    if (NULL == hw_module) {
        ALOGE("No valid fingerprint module");
        return 0;
    }

    mModule = reinterpret_cast<const fingerprint_module_t*>(hw_module);

    if (mModule->common.methods->open == NULL) {
        ALOGE("No valid open method");
        return 0;
    }

    hw_device_t *device = NULL;

    if (0 != (err = mModule->common.methods->open(hw_module, NULL, &device))) {
        ALOGE("Can't open fingerprint methods, error: %d", err);
        return 0;
    }

    if (kVersion != device->version) {
        ALOGE("Wrong fp version. Expected %d, got %d", kVersion, device->version);
        // return 0; // FIXME
    }

    mDevice = reinterpret_cast<fingerprint_device_t*>(device);
    err = mDevice->set_notify(mDevice, hal_notify_callback);
    if (err < 0) {
        ALOGE("Failed in call to set_notify(), err=%d", err);
        return 0;
    }

    // Sanity check - remove
    if (mDevice->notify != hal_notify_callback) {
        ALOGE("NOTIFY not set properly: %p != %p", mDevice->notify, 
        hal_notify_callback);
    }

    ALOG(LOG_VERBOSE, LOG_TAG, "fingerprint HAL successfully initialized");
    return reinterpret_cast<int64_t>(mDevice); // This is just a handle
}

openHal() will call the hw_get_module() to get the pointer to hw_module_t module, after then it will call the open() function. Once the HAL module is opened, the Fingerprintd is able to operate fingerprint device through the hw_device_t.

The funciton of the fingerprint module can be found at fingerprint.h and fingerprint.c.

For now, we have gone over the whole process of the fingerprint working. we can give the summary here.

ServiceManager->FingerprintService.java->FingerprintDaemonProxy.cpp->fingerprint.c
->vendorHal.cpp->vendorCA.cpp——–TEE->TA.c

However, from Android 8.0, Android has made some changes to the HAL access method and introduced the HIDL concept. Therefore the contents we introduced are for the Android early version before 8.0.

Next, I will write an article to introduce the difference of the fingerprint framework on the Android 8.0 version.

Android Fingerprint Framework (2)

David Dong — Sat, 07 Dec 2019 23:52:01 +0800

This page will follow the last article to continue introducing the android fingerprint framework knowledge. The content is focused on android source code inspecting and analysis.

Step one - startup fingerprintd service

Looking at the init.rc file, a task is assigned at init.rc when the android system boots up - start the fingerprint daemon service.

service fingerprintd /system/bin/fingerprintd
class late_start
user root
group root sdcard_r sdcard_rw

Let’s go on to check the fingerprintd program.
Here I would recommend a useful website for you viewing the android source code.
Android Community

We can see the android path of the fingerprintd.cpp is system/core/fingerprintd/ and the directory structure is as below.

read the Android.mk

androdi path: root/system/core/fingerprintd/Android.mk

we can know that this package is built as a executable program.

LOCAL_PATH := $(call my-dir)
include $(CLEAR_VARS)
LOCAL_CFLAGS := -Wall -Wextra -Werror -Wunused
LOCAL_SRC_FILES := \
	FingerprintDaemonProxy.cpp \
	IFingerprintDaemon.cpp \
	IFingerprintDaemonCallback.cpp \
	fingerprintd.cpp
LOCAL_MODULE := fingerprintd
LOCAL_SHARED_LIBRARIES := \
	libbinder \
	liblog \
	libhardware \
	libutils \
	libkeystore_binder
include $(BUILD_EXECUTABLE)

next open the fingerprintd.cpp

android path: root/system/core/fingerprintd/fingerprintd.cpp

The task of the main() function is very simple, just create a FingerprintDaemonProxy object and add it into the service queue.

#include "FingerprintDaemonProxy.h"

int main() {
    ALOGI("Starting " LOG_TAG);
    android::sp<android::IServiceManager> serviceManager 
    = android::defaultServiceManager();
    android::sp<android::FingerprintDaemonProxy> proxy =
            android::FingerprintDaemonProxy::getInstance();
    android::status_t ret = serviceManager->addService(
            android::FingerprintDaemonProxy::descriptor, proxy);
    if (ret != android::OK) {
        ALOGE("Couldn't register " LOG_TAG " binder service!");
        return -1;
    }

    /*
     * We're the only thread in existence, so we're just going to process
     * Binder transaction as a single-threaded program.
     */
    android::IPCThreadState::self()->joinThreadPool();
    ALOGI("Done");
    return 0;
}

From the FingerprintDaemonProxy.h

android path: root/system/core/fingerprintd/

FingerprintDaemonProxy.h
We find the remote service is fingerprint daemon. Fingerprinted registers the remote service to the servicemanager for the client to use. The protocol interface is IFingerprintdaemon. FingerprintService in the framework will eventually call the remote service, that is, the method in fingerprintdaemonproxy.cpp.

android path: root/system/core/fingerprintd/

fingerprintdaemonproxy.cpp

++
#ifndef FINGERPRINT_DAEMON_PROXY_H_
#define FINGERPRINT_DAEMON_PROXY_H_

#include "IFingerprintDaemon.h"
#include "IFingerprintDaemonCallback.h"

namespace android {

class FingerprintDaemonProxy : public BnFingerprintDaemon {
    public:
        static FingerprintDaemonProxy* getInstance() {
            if (sInstance == NULL) {
                sInstance = new FingerprintDaemonProxy();
            }
            return sInstance;
        }

        // These reflect binder methods.
        virtual void init(const sp<IFingerprintDaemonCallback>& callback);
        virtual int32_t enroll(const uint8_t* token, ssize_t 
        tokenLength, int32_t groupId, int32_t timeout);
        virtual uint64_t preEnroll();
        virtual int32_t postEnroll();
        virtual int32_t stopEnrollment();
        virtual int32_t authenticate(uint64_t sessionId, uint32_t 
        groupId);
        virtual int32_t stopAuthentication();
        virtual int32_t remove(int32_t fingerId, int32_t groupId);
        virtual int32_t enumerate();
        virtual uint64_t getAuthenticatorId();
        virtual int32_t setActiveGroup(int32_t groupId, 
        const uint8_t* path, ssize_t pathLen);
        virtual int64_t openHal();
        virtual int32_t closeHal();

    private:
        FingerprintDaemonProxy();
        virtual ~FingerprintDaemonProxy();
        void binderDied(const wp<IBinder>& who);
        void notifyKeystore(const uint8_t *auth_token, const size_t
        auth_token_length);
        static void hal_notify_callback(const fingerprint_msg_t *msg);

        static FingerprintDaemonProxy* sInstance;
        fingerprint_module_t const* mModule;
        fingerprint_device_t* mDevice;
        sp<IFingerprintDaemonCallback> mCallback;
};

} // namespace android

#endif // FINGERPRINT_DAEMON_PROXY_H_

Step two - Startup FingerprintService

Next, we will move to the framework layer to find how the Fingerprint Service starts up. open the SystemServer.java

android path: root/frameworks/base/services/java/com/android/server/

SystemServer.java
This class is in charge of system service management, include start-up the necessary service. When the Android system loads the system server, starts Fingerprint Service.

import com.android.server.fingerprint.FingerprintService;

if (mPackageManager.hasSystemFeature(PackageManager.FEATURE_FINGERPRINT)) {
                traceBeginAndSlog("StartFingerprintSensor");
                mSystemServiceManager.startService(FingerprintService.class);
                traceEnd();
            }

Keep looking into the FingerprintService.java.

android path: root/frameworks/base/services/core/java/com/android/server/fingerprint/FingerprintService.java

FingerprintService is a subclass of SystemService class and implements the IHwbinder interface.

public class FingerprintService extends SystemService implements
IHwBinder.DeathRecipient {

    public synchronized IBiometricsFingerprint getFingerprintDaemon() {
        if (mDaemon == null) {
            Slog.v(TAG, "mDaemon was null, reconnect to fingerprint");
            try {
                mDaemon = IBiometricsFingerprint.getService();
            } catch (java.util.NoSuchElementException e) {
                // Service doesn't exist or cannot be opened. Logged below.
            } catch (RemoteException e) {
                Slog.e(TAG, "Failed to get biometric interface", e);
            }
            if (mDaemon == null) {
                Slog.w(TAG, "fingerprint HIDL not available");
                return null;
            }

            mDaemon.asBinder().linkToDeath(this, 0);

            try {
                mHalDeviceId = mDaemon.setNotify(mDaemonCallback);
            } catch (RemoteException e) {
                Slog.e(TAG, "Failed to open fingerprint HAL", e);
                mDaemon = null; // try again later!
            }

            if (DEBUG) Slog.v(TAG, "Fingerprint HAL id: " + mHalDeviceId);
            if (mHalDeviceId != 0) {
                loadAuthenticatorIds();
                updateActiveGroup(ActivityManager.getCurrentUser(), null);
                doFingerprintCleanupForUser(ActivityManager.getCurrentUser());
            } else {
                Slog.w(TAG, "Failed to open Fingerprint HAL!");
                MetricsLogger.count(mContext, "fingerprintd_openhal_error", 1);
                mDaemon = null;
            }
        }
        return mDaemon;
    }

}

Let’s see the method getFingerprintDaemon(), this method will acquire the fingerprint remote service object, that is, the object of fingerprint daemon (system/core/fingerprintd), which has been registered in the init.rc. Then initialize the remote service fingerprintdaemon and set the callback mDaemonCallback.

It can be seen from the above that the fingerprint service in the framework calls the fingerprint remote service of the native layer fingerprint daemon (related to the hardware), which can be regarded as the client of the fingerprint remote service fingerprint daemon.

Ok, we have already gone through the working process of the framework layer and how they register the system service and access the HAL code by calling the remote Fingerprint Service through Binder. Let’s move to the native layer in the next article.

Android Fingerprint Framework (1)

David Dong — Thu, 03 Oct 2019 15:17:47 +0800

This page is trying to present a brief introduction on the android fingerprint framework, aimed at helping anyone who wants to learn android fingerprint-related knowledge, created by referring to some documents and android source code.

There are some articles throughout the internet to share android fingerprint-related knowledge. However, from my point of view, part of them are too complicated to understand and some aren’t very distinct. I wrote this article by referring to android source code and according to my personal working experiences, tried to make the contents simple and easy to understand.

From Android 6.0 (API23), android introduced a fingerprint API interface, that is FingerprintManager to support fingerprint identification. While at android 8.0 (API26), it is deprecated and replaced by BiometricPrompt API interface.

The below description is from the android official document about the fingerprint framework.

Architecture

The Fingerprint HAL interacts with the following components.

BiometricManager interacts directly with an app in an app process. Each app has an instance of IBiometricsFingerprint.hal

FingerprintService operates in the system process, which handles communication with fingerprint HAL.

Fingerprint HAL is a C/C++ implementation of the IBiometricsFingerprint HIDL interface. This contains the vendor-specific library that communicates with the device-specific hardware.

Keystore API and Keymaster components provide hardware-backed cryptography for secure key storage in a secure environment, such as the Trusted Execution Environment (TEE).
A vendor-specific HAL implementation must use the communication protocol required by a TEE. Raw images and processed fingerprint features must not be passed in untrusted memory. All such biometric data needs to be stored in secure hardware such as the TEE. Rooting must not be able to compromise biometric data.
FingerprintService and fingerprintd make calls through the Fingerprint HAL to the vendor-specific library to enroll fingerprints and perform other operations.

Implementation guidelines

The following Fingerprint HAL guidelines are designed to ensure that fingerprint data is not leaked and is removed when a user is removed from a device:

Raw fingerprint data or derivatives (for example, templates) must never be accessible from outside the sensor driver or TEE. If the hardware supports a TEE, hardware access must be limited to the TEE and protected by an SELinux policy. The Serial Peripheral Interface (SPI) channel must be accessible only to the TEE and there must be an explicit SELinux policy on all device files.
Fingerprint acquisition, enrollment, and recognition must occur inside the TEE.
Only the encrypted form of the fingerprint data can be stored on the file system, even if the file system itself is encrypted.
Fingerprint templates must be signed with a private, device-specific key. For Advanced Encryption Standard (AES), at a minimum, a template must be signed with the absolute file-system path, group, and finger ID such that template files are inoperable on another device or for anyone other than the user that enrolled them on the same device. For example, copying fingerprint data from a different user on the same device or another device must not work.
Implementations must either use the file-system path provided by the setActiveGroup() function or provide a way to erase all user template data when the user is removed. It’s strongly recommended that fingerprint template files be stored as encrypted and stored in the path provided. If this is infeasible due to TEE storage requirements, the implementer must add hooks to ensure the removal of the data when the user is removed.

Working process

By referring to the official introduction, here I summarized the workflow according to my understanding.

Android starts the fingerprint daemons process-Fingerprintd during boot up in init.rc.
System server will start the fingerprint system service Fingerprint Service.
Fingerprint Service calls the interface of Fingerprintd to communicate with the fingerprint HAL code.
Fingerprint HAL interacts with the TA program and the latter undertakes the communication with fingerprint hardware through SPI.
For the sake of security, the hardware SPI is mounted in the TEE environment. The data collection of fingerprint images and the related processing of the algorithm are all carried out within the TEE environment. The REE side only obtains the result of the TEE side.

In the next article, I will make a detailed introduction to the work process by inspecting the android source code.

To be continued…

Fingerprint enrollment in Android

David Dong — Sun, 15 Sep 2019 22:14:47 +0800

When we do fingerprint enrollment in an android device, it will generate a fingerprint template that contains the user’s fingerprint information in a specified directory (for example, /data/vendor_de/0/fpdata/user.db), which depends on the system configuration.

During the template is being generated, there is a series of APIs calling in the android system internally. Let’s have a look at this process.

Table of content

preEnroll()
enroll()
postEnroll()

Android sets some standard APIs in HAL layer for fingerprint event handling. Refer to Android P source code at BiometricsFingerprint.h and BiometricsFingerprint.cpp.

The fingerprint APIs definition in Android BiometricsFingerprint.h.

Return<uint64_t> setNotify(const sp<IBiometricsFingerprintClientCallback>& clientCallback) override;
Return<uint64_t> preEnroll() override;
Return<RequestStatus> enroll(const hidl_array<uint8_t, 69>& hat, uint32_t gid, uint32_t timeoutSec) override;
Return<RequestStatus> postEnroll() override;
Return<uint64_t> getAuthenticatorId() override;
Return<RequestStatus> cancel() override;
Return<RequestStatus> enumerate() override;
Return<RequestStatus> remove(uint32_t gid, uint32_t fid) override;
Return<RequestStatus> setActiveGroup(uint32_t gid, const hidl_string& storePath) override;
Return<RequestStatus> authenticate(uint64_t operationId, uint32_t gid) override;

preEnroll()

When a user starts to enroll the fingerprint, the method preEnroll() will be called firstly, the fingerprint IC vendor needs to override this method to complete the below task.

generates and saves a 64-bit random number (challenge) in the fingerprint TA. This random number has two uses:
1. Return to the REE upper layer to fill in the authenticated token challenge in the enrollment.
2. TA will use it to preliminarily verify the next enrollment to ensure that the enrollment has not been tampered with by a third party.

enroll()

If the preEnroll() is proper returned, enroll() will be following called.
Let’s see the definition of enroll().

int (*enroll)(struct fingerprint_device *dev, const hw_auth_token_t *hat,
          uint32_t gid, uint32_t timeout_sec);

parameters:

hw_auth_token_t: this struct encloses the tokens of enrollment.

/**
 * Data format for an authentication record used to prove successful authentication.
 */
typedef struct __attribute__((__packed__))
{
    uint8_t version;  // Current version is 0
    uint64_t challenge;
    uint64_t user_id;             // secure user ID, not Android user ID
    uint64_t authenticator_id;    // secure authenticator ID
    uint32_t authenticator_type;  // hw_authenticator_type_t, in network order
    uint64_t timestamp;           // in network order
    uint8_t hmac[32];
} hw_auth_token_t;

version: version number of this token.
challenge: it is the 64 bit random number to which preenroll was previously called to prevent the enroll from being counterfeited by a third party this time.
user_id: Security ID, not Android user ID.
authenticator_id: used to indicate different authentication permissions.Normally it indicates the fingerprint database ID.
authenticator_type: 0x00 for gatekeeper, 0x01 for fingerprint.
timestamp: last boot time stamp.
hmac: a special key and SHA-256 algorithm are used to calculate the previous batch of parameters, and a HMAC value is obtained to ensure the legitimacy and security of the previous parameters.

gid: Indicate which user enroll fingerprint (Android supports multiple users). The gid is used to set the store path of the fingerprint template on REE side.
timeout_sec: timeout /second.

The upper layer calls enroll() method and passes the parameters to fingerprint TA, fingerprint TA receives the token and authorizes the token. There will be some works.

check if the received token->challenge same as the previous preEnroll phase challenge.
check if the token version is the same.
check if authenticator_type is the same.
retrieve the data of hw_auth_token_t struct and calculate the HMAC, check if the calculated value is the same as the original.

Once the token is authorized then the fingerprint TA switches the fingerprint sensor status to ready for enrollment and the fingerprint sensor will wait for the finger-down event.

If the fingerprint sensor detects the finger down occurs, it will trigger the interrupt to inform the fingerprint TA to capture the image. Fingerprint TA will capture the image and do verification to check if the image is qualified. Uses on_acquired() callback method to notify the REE upper layer of the result. If the image is not good, it will repeat the image capture for a pre-setted time. If the image is qualified, then fingerprint TA will start the enrollment in ALGO and call on_enroll_result() to notify the upper layer of the remained sample times. Waiting for the finger up and do the next round of image capture. This loop will be repeated until the required count of images is achieved.

postEnroll()

Once the enroll() is returned, postEnroll() will be called to finish one-time enrollment process. the work of postEnroll() is to update the challenge of TA.

The simplest diagram of the whole enrollment process is

enrollment APIs calling.

preEnroll()->enroll()->preEnroll()

enroll() execute flow.

enroll()->authorize token->wait_for_finger_down->capture_image->algo_enroll->notify->repeat->remain sample times?->update_template->end_ernoll

Some tips for android development errors

David Dong — Fri, 26 Jul 2019 23:45:19 +0800

Today I met an error when I was trying to customize the android.widget.Button Class.😕
The error information is

"This custom view should extend android.support.v7.widget.AppCompatButton instead."

The temporary solution that I used is as below.

import android.annotation.SuppressLint;
@SuppressLint("AppCompatCustomView")

Besides, I had met

"I/hwservicemanager: getTransport: Cannot find entry"

when running the application. That’s because the application used the HIDL interface which should register at the vintf/manifest.xml but hasn’t been done. This feature is introduced at Android O. Obviously, the solution is registering all of the HIDL interfaces that will be used at manifest.xml.

Another issue is

"avc: denied { read } for name="u:object_r:vendor_default_prop:s0" dev="tmpfs" ino=27157 
scontext=u:r:platform_app:s0:c512,c768 tcontext=u:object_r:vendor_default_prop:s0 
tclass=file permissive=1"

which is related to selinux, the selinux policy is missing that cause the permission is rejected. Need to add the permission to selinux.

Python re module

David Dong — Sat, 11 May 2019 20:36:23 +0800

re module is a unique string matching module of Python.

Many functions provided in this module are implemented based on regular expressions.

The module is handy, to use it, need to import re module at first.

prototype

re.split(pattern, string, maxsplit=0, flags=0)

parameters

pattern: Separator, can be character or regular expression.
string: String to split.
maxsplit: Maximum number of split. default is 0, which means no limitation. if minus, means no split.
flags: Used to control how regular expressions are matched.

examples

split the input string by space and ,, only do one time split.

import re
inputs = re.split(r'[\s,]+', input("Enter: 'ID' ['File name'] >> "), 1)
Enter: 'ID' ['File name'] >> 01 filename
inputs
['01', 'filename']
inputs = re.split(r'[\s,]+', input("Enter: 'ID' ['File name'] >> "), 1)
Enter: 'ID' ['File name'] >> 01,filename
inputs
['01', 'filename']
inputs = re.split(r'[\s,]+', input("Enter: 'ID' ['File name'] >> "), 1)
Enter: 'ID' ['File name'] >> 01,file1,file2 file3
inputs
['01', 'file1,file2 file3']

split the string by character.

import re
s = 'ab12,% ffa[]-(fceknl)'
re.split(r'b',s)
['a', '12,% ffa[]-(fceknl)']
re.split(r'f',s)
['ab12,% ', '', 'a[]-(', 'ceknl)']

match multiple characters

import re
s = 'ab12,% ffa[]-(fceknl)'
re.split(r'[1,en]',s)
['ab', '2', '% ffa[]-(fc', 'k', 'l)']
re.split(r'[-e(n]',s)
['ab12,% ffa[]', '', 'fc', 'k', 'l)']

keep separator

import re
s = 'ab12,% ffa[]-(fceknl)'
re.split('([af])',s)
['', 'a', 'b12,% ', 'f', '', 'f', '', 'a', '[]-(', 'f', 'ceknl)']

Python struct.pack()

David Dong — Fri, 03 May 2019 14:07:05 +0800

Python uses struct.pack() to pack python data type into binary data (Byte Stream), which means convert a value to a string based on the format.

Because there is no byte type in Python, where the string is equivalent to a byte stream, or byte array.

struct.pack()

The function prototype:

struck.pack(format, data1, data2,...)

format: define the converted format. data1,data2,…: the data needs to be packed.

The format includes

Format	C Type	Python Type	Standard Size
x	pad byte
c	char	string of length 1	1
b	signed char	integer	1
B	unsigned char	integer	1
?	_bool	bool	1
h	short	integer	2
H	unsigned short	integer	2
i	int	integer	4
I	unsigned int	integer	4
l	long	integer	4
L	unsigned long	integer	4
q	long long	integer	8
Q	unsigned long long	integer	8
f	float	float	4
d	double	float	8
s	char[]	string
p	char[]string
P	void *	integer

To exchange data with the structure in C, it is also necessary to consider that some C or C + + compilers use byte alignment, usually a 32-bit system with 4 bytes as the unit. Therefore, a struct can be converted according to the byte order of the local machine. The alignment can be changed by the first character in the format. The definition is as follows:

Format	Byte Order	Size	alignment
@	native byte	native	native
=	native	standard	none
<	little-endian char	standard	none
>	big-endian char	standard	none
!	network (= big-endian)	standard	none

Use it in the first place of format.

for example.

import struct

a = 11
b = 12

print(len(struct.pack("ii",a,b)))
8
print(struct.pack("ii",a,b))
b'\x0b\x00\x00\x00\x0c\x00\x00\x00'

struct.unpack()

struct.unpack() unpacks the byte stream into Python data type.
The function prototype:

struct.unpack(fmt, string)

This function return a tuple.

For example.

a = 11
b = 12
packdata = struct.pack("ii",a,b)
c,d = struct.unpack("1i1i",packdata)
print((c,d))
(11,12)

struct.calcsize()

Struct.calcsize() is used to calculate the length of the result corresponding to the format string.

For example.

print(struct.calcsize("c"))
1
print(struct.calcsize("H"))
2
print(struct.calcsize("L"))
4
print(struct.calcsize("Q"))
8

A quick way to hide app icon on andorid desktop

David Dong — Mon, 22 Apr 2019 19:11:12 +0800

Here introduces a quick way to hide the android app icon on the desktop. Although it isn’t a common case on the development or usage of android app, it deserves pay a little bit time reading for anyone who want to realize it.

The code implementation

The method is to add a <data></data> property on your app’s AndroidManifest.xml.
Below is a example.

<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:tools="http://schemas.android.com/tools"
    package="com.example.hideapplication02">

    <application
        android:allowBackup="true"
        android:icon="@mipmap/ic_launcher"
        android:label="@string/app_name"
        android:roundIcon="@mipmap/ic_launcher_round"
        android:supportsRtl="true"
        android:theme="@style/Theme.HideApplication02">
        <activity
            android:name=".MainActivity"
            android:exported="true">
            <intent-filter>
                <action android:name="android.intent.action.MAIN" />
				<!-- hide application icon start -->
                <data android:host="mainactivity" android:scheme="com.example.hideapplication02"/>
                <category android:name="android.intent.category.LAUNCHER" />
            </intent-filter>
        </activity>
    </application>

</manifest>

Comments

The key points are:

App’s entry activity is declared to be started by receiving implicit intent, so the icon will not be displayed naturally.
For the data property declaring, the android: scheme must have been defined explicitly.
Property value needs to start with a lowercase letter.
To call the app, need to use URI, the formate is scheme://host: port/path.
Therefore, for above example, the URI is com.example.hideapplication02://mainactivity.
The caller needs to add below code:

  Uri uri = Uri.parse("com.example.hideapplication02://mainactivity");
  Intent intent = new Intent(null, uri);
  startActivity(intent);

Python 开发编码规范（转）

David Dong — Mon, 01 Apr 2019 22:17:33 +0800

这篇文档所给出的编码约定适用于在主要的Python发布版本中组成标准库的Python 代码，请查阅相关的关于在Python的C实现中C代码风格指南的描述。

这篇文档改编自Guido最初的《Python风格指南》一文，并从《Barry’s style guide》中添加了部分内容。在有冲突的地方，Guido的风格规则应该是符合本PEP的意图(译注：指当有冲突时，应以Guido风格为准)。这篇PEP仍然尚未完成(实际上，它可能永远都不会完成)。

在这篇风格指导中的一致性是重要的。在一个项目内的一致性更重要。在一个模块或函数内的一致性最重要。但最重要的是：知道何时会不一致——有时只是没有实施风格指导。当出现疑惑时，运用你的最佳判断，看看别的例子，然后决定怎样看起来更好。并且要不耻下问！打破一条既定规则的两个好理由：

当应用这个规则是将导致代码可读性下降，即便对某人来说，他已经习惯于按这条规则来阅读代码了。
为了和周围的代码保持一致而打破规则(也许是历史原因)，虽然这也是个清除其它混乱的好机会(真正的XP风格)。

好了，闲话少说，正文如下。

1. 代码的布局

1.1 缩进

使用Emacs的Python-mode的默认值：4个空格一个缩进层次。对于确实古老的代码，你不希望产生混乱，可以继续使用8空格的制表符(8-space tabs)。Emacs Python-mode自动发现文件中主要的缩进层次，依此设定缩进参数。

1.2 制表符还是空格

永远不要混用制表符和空格。最流行的Python缩进方式是仅使用空格，其次是仅使用制表符，混合着制表符和空格缩进的代码将被转换成仅使用空格。(在Emacs中，选中整个缓冲区，按ESC-x去除制表符。)调用Python命令行解释器时使用-t选项，可对代码中不合法得混合制表符和空格发出警告，使用-tt时警告将变成错误。这些选项是被高度推荐的。对于新的项目，强烈推荐仅使用空格而不是制表符。许多编辑器拥有使之易于实现的功能(在Emacs中，确认indent-tabs-mode是nil)。

1.3 行的最大长度

周围仍然有许多设备被限制在每行80字符：而且，窗口限制在80个字符。使将多个窗口并排放置成为可能。在这些设备上使用默认的折叠方式看起来有点丑陋。因此，请将所有行限制在最大79字符(Emacs准确得将行限制为长80字符)，对顺序排放的大块文本(文档字符串或注释)，推荐将长度限制在72字符。折叠长行的首选方法是使用Pyhon支持的圆括号，方括号和花括号内的行延续。如果需要，你可以在表达式周围增加一对额外的圆括号，但是有时使用反斜杠看起来更好，确认恰当得缩进了延续的行。 Emacs的Python-mode正确得完成了这些。一些例子：

#!Python
class Rectangle(Blob):
def __init__(self,width,height,color='black',emphasis=None,highlight=0):
if width == 0 and height == 0 and \
color == 'red' and emphasis == 'strong' or \
highlight > 100:
raise ValueError, "sorry， you lose"
if width == 0 and height == 0 and (color == 'red' or
emphasis is None):
raise ValueError,"I don't think so"
Blob.__init__(self,width,height,color,emphasis,highlight)

1.4 空行

用两行空行分割顶层函数和类的定义，类内方法的定义用单个空行分割，额外的空行可被用于(保守的)分割相关函数组成的群，在一组相关的单句中间可以省略空行。(例如：一组哑元素)。当空行用于分割方法的定义时，在class行和第一个方法定义之间也要有一个空行。在函数中使用空行时，请谨慎的用于表示一个逻辑段落。Python接受contol-L(即^L)换页符作为空格：Emacs(和一些打印工具)，视这个字符为页面分割符，因此在你的文件中，可以用他们来为相关片段分页。

2. 编码

Python核心发布中的代码必须始终使用ASCII或Latin-1编码(又名 ISO-8859-1)，使用ASCII的文件不必有编码cookie，Latin-1仅当注释或文档字符串涉及作者名字需要Latin-1时才被使用：另外使用\x转义字符是在字符串中包含非ASCII(non-ASCII)数据的首选方法。作为PEP 263实现代码的测试套件的部分文件是个例外。

2.1 导入

通常应该在单独的行中导入(Imports)，例如：

No： import sys， os
Yes：import sys
import os

但是这样也是可以的：
from types import StringType， ListType
Imports 通常被放置在文件的顶部，仅在模块注释和文档字符串之后，在模块的全局变量和常量之前。
Imports应该有顺序地成组安放：

标准库的导入(Imports )
相关的主包(major package)的导入(即，所有的email包在随后导入)
特定应用的导入(imports) 你应该在每组导入之间放置一个空行，对于内部包的导入是不推荐使用相对导入的，对所有导入都要使用包的绝对路径。
从一个包含类的模块中导入类时，通常可以写成这样：
from MyClass import MyClass
from foo.bar.YourClass import YourClass
如果这样写导致了本地名字冲突，那么就这样写 import MyClass
import foo.bar.YourClass
即使用MyClass.MyClass和foo.bar.YourClass.YourClass

2.2 表达式和语句中的空格

Guido不喜欢在以下地方出现空格：

紧挨着圆括号，方括号和花括号的；
如：spam( ham[ 1 ]，{ eggs：2 } )。要始终将它写成spam(ham[1]，{eggs： 2})。
紧贴在逗号，分号或冒号前的；
如： if x == 4：print x，y：x，y = y，x。要始终将它写成 if x == 4:print x，y:x，y = y，x。
紧贴着函数调用的参数列表前开式括号(open parenthesis )的；
如spam (1)。要始终将它写成spam(1)。
紧贴在索引或切片，开始的开式括号前的；
如： dict ['key'] = list [index]。要始终将它写成dict['key'] = list[index]。
在赋值(或其它)运算符周围的用于和其它并排的一个以上的空格；
如：

#!Python
x= 1
y= 2
long_variable = 3

要始终将它写成

#!Python
x = 1
y = 2
long_variable = 3

2.3 其他建议

始终在这些二元运算符两边放置一个空格：赋值(=)，比较(==，<，>，!=，<>，<=， >=，in，not in，is，is not)，布尔运算 (and，or，not)。按你的看法在算术运算符周围插入空格。始终保持二元运算符两边空格的一致。
一些例子：

#!Python
i = i+1
submitted = submitted + 1
x = x*2 - 1
hypot2 = x*x + y*y
c = (a+b) * (a-b)
c = (a + b) * (a - b)

不要在用于指定关键字参数或默认参数值的=号周围使用空格，例如：

#!Python
def complex(real, imag=0.0):
return magic(r=real, i=imag)

不要将多条语句写在同一行上：

No: if foo == 'blah':do_blah_thing()
Yes:if foo == 'blah':
do_blah_thing()
No:do_one():do_two():do_three()
Yes: do_one()
do_two()
do_three()

3. 注释

同代码不一致的注释比没注释更差。当代码修改时，始终优先更新注释 !注释应该是完整的句子，如果注释是一个短语或句子，首字母应该大写，除非他是一个以小写字母开头的标识符(永远不要修改标识符的大小写)。如果注释很短，最好省略末尾的句号。注释块通常由一个或多个由完整句子构成的段落组成，每个句子应该以句号结尾。你应该在句末，句号后使用两个空格，以便使Emacs的断行和填充工作协调一致。用英语书写时，断词和空格是可用的。非英语国家的Python程序员：请用英语书写你的注释，除非你120%的确信这些代码不会被不懂你的语言的人阅读。注释块注释块通常应用于跟随着一些(或者全部)代码并和这些代码有着相同的缩进层次。注释块中每行以#和一个空格开始(除非他是注释内的缩进文本)。注释块内的段落以仅含单个#的行分割。注释块上下方最好有一空行包围(或上方两行下方一行，对一个新函数定义段的注释)。行内注释一个行内注释是和语句在同一行的注释，行内注释应该谨慎适用，行内注释应该至少用两个空格和语句分开，它们应该以#和单个空格开始。

x = x+1 # Increment x

如果语意是很明了的，那么行内注释是不必要的，事实上是应该被移除的。不要这样写：

x = x+1 # Increment x
x = x+1 # Compensate for border

但是有时，这样是有益的：

x = x+1 # Compensate for border

3.1 文档字符串

应该一直遵守编写好的文档字符串的约定PEP 257 [3]。为所有公共模块，函数，类和方法编写文档字符串。文档字符串对非公开的方法不是必要的，但你应该有一个描述这个方法做什么的注释。这个注释应该在def这行后。 PEP 257 描述了好的文档字符串的约定。一定注意，多行文档字符串结尾的"""应该单独成行，例如：

"""Return a foobang
Optional plotz says to frobnicate the bizbaz first。
"""

对单行的文档字符串，结尾的"""在同一行也可以。版本注记如果你要将RCS或CVS的杂项(crud)包含在你的源文件中，按如下做。

#!Python
__version__ = "$Revision： 1。4 $"
# $Source： E：/cvsroot/Python_doc/pep8。txt，v $

这个行应该包含在模块的文档字符串之后，所有代码之前，上下用一个空行分割。

4. 命名约定

Python库的命名约定有点混乱，所以我们将永远不能使之变得完全一致，不过还是有公认的命名规范的。新的模块和包(包括第三方的框架)必须符合这些标准，但对已有的库存在不同风格的，保持内部的一致性是首选的。

4.1 描述：命名风格

有许多不同的命名风格。以下的有助于辨认正在使用的命名风格，独立于它们的作用。以下的命名风格是众所周知的：

b (单个小写字母)
B (单个大写字母)
Lowercase（小写）
lower_case_with_underscores（有下划线的小写）
UPPERCASE（大写）
UPPER_CASE_WITH_UNDERSCORES（有下划线的大写）
CapitalizedWords (或 CapWords，CamelCase这样命名是因为可从字母的大小写分出单词。这有时也被当作StudlyCaps。
mixedCase (与CapitalizedWords的不同在于首字母小写!)
Capitalized_Words_With_Underscores（有下划线的首字母大写） (丑陋!)
还有用短的特别前缀将相关的名字聚合在一起的风格。这在Python中不常用，但是出于完整性要提一下，例如，os.stat()函数返回一个元组，他的元素传统上说名如st_mode， st_size，st_mtime等等。 X11库的所有公开函数以X开头。(在Python中，这个风格通常认为是不必要的，因为属性和方法名以对象作前缀，而函数名以模块名作前缀。)

另外，以下用下划线作前导或结尾的特殊形式是被公认的(这些通常可以和任何习惯组合)：

_single_leading_underscore(单个下划线作前导)：弱的“内部使用(internal use)”标志。 (例如，from M import *不会导入以下划线开头的对象)。
single_trailing_underscore_(单个下划线结尾)：用于避免与Python关键词的冲突，例如：Tkinter.Toplevel(master，class_='ClassName')。
_double_leading_underscore(双下划线)：从Python 1.4起为类私有名。
_double_leading_and_trailing_underscore_：magic对象或属性，存在于用户控制的(user-controlled)名字空间，例如：_init_， _import_ 或_file_。有时它们被用户定义用于触发某个魔法行为(例如：运算符重载)：有时被构造器插入，以便自己使用或为了调试。因此，在未来的版本中，构造器(松散得定义为Python解释器和标准库)可能打算建立自己的魔法属性列表，用户代码通常应该限制将这种约定作为己用。
欲成为构造器的一部分的用户代码可以在下滑线中结合使用短前缀，例如： _bobo_magic_attr__。

4.2 说明：命名约定

应避免的名字。永远不要用字符l(小写字母el(就是读音，下同))，O(大写字母oh)，或I(大写字母eye)作为单字符的变量名。在某些字体中这些字符不能与数字1和0分辨。试着在使用l时用L代替。

4.3 模块名

模块应该是不含下划线的，简短的，小写的名字。因为模块名被映射到文件名，有些文件系统大小写不敏感并且截短长名字，模块名被选为相当短是重要的，这在Unix上不是问题，但当代码传到Mac或Windows上就可能是个问题了。当用C或C++编写的扩展模块有一个伴随Python模块提供高层(例如进一步的面向对象)接口时，C/C++模块有下划线前导(如：_socket)。Python包应该是不含下划线的，简短的，全小写的名字。

4.4 类名

几乎不出意料，类名使用CapWords约定。内部使用的类外加一个前导下划线。

4.5 异常名

如果模块对所有情况定义了单个异常，它通常被叫做error或Error。似乎内建(扩展)的模块使用error(例如：os.error)，而Python模块通常用Error (例如：xdrlib.Error)。趋势似乎是倾向使用CapWords异常名。

4.6 全局变量名

(让我们祈祷这些变量仅在一个模块的内部有意义) 这些约定和在函数中的一样。模块是被设计为通过from M import *来使用的，必须用一个下划线作全局变量(及内部函数和类)的前缀防止其被导出(exporting)。

4.7 函数名

函数名应该为小写，可能用下划线风格单词以增加可读性。mixedCase仅被允许用于这种风格已经占优势的上下文(如：threading.py)，以便保持向后兼容。

4.8 方法名和实例变量

这段大体上和函数相同：通常使用小写单词，必要时用下划线分隔增加可读性。仅为不打算作为类的公共界面的内部方法和实例使用一个前导下划线，Python不强制要求这样：它取决于程序员是否遵守这个约定。使用两个前导下划线以表示类私有的名字，Python将这些名字和类名连接在一起：如果类Foo有一个属性名为_a，它不能以Foo._a访问。(固执的用户还是可以通过Foo._Foo__a得到访问权。) 通常双前导下划线仅被用于避免含子类的类中的属性名的名字冲突。

5. 继承的设计

始终要确定一个类中的方法和实例变量是否要被公开。通常，永远不要将数据变量公开，除非你实现的本质上只是记录，人们几乎总是更喜欢代之给出一个函数作为类的界面(Python 2.2 的一些开发者在这点上做得非常漂亮)。同样，确定你的属性是否应为私有的。私有和非私有的区别在于模板将永远不会对原有的类(导出类)有效，而后者可以。你应该在大脑中就用继承设计好了你的类，私有属性必须有两个前导下划线，无后置下划线，非公有属性必须有一个前导下划线，无后置下划线，公共属性没有前导和后置下划线，除非它们与保留字冲突，在此情况下，单个后置下划线比前置或混乱的拼写要好，例如：class_优于klass。最后一点有些争议：如果相比class_你更喜欢klass，那么这只是一致性问题。

5.1 设计建议

单个元素(singletons)的比较，如None 应该永远用：is或is not来做。当你本意是if x is not None时，对写成if x要小心。例如当你测试一个默认为None的变量或参数是否被设置为其它值时，这个值也许在布尔上下文(Boolean context)中是false！基于类的异常总是好过基于字符串的异常。模块和包应该定义它们自己的域内特定的基异常类，基类应该是内建的Exception类的子类。还始终包含一个类的文档字符串。例如：

#!Python
class MessageError(Exception):
"""Base class for errors in the email package。"""

使用字符串方法(methods)代替字符串模块，除非必须向后兼容Python 2.0以前的版本。字符串方法总是非常快，而且和unicode字符串共用同样的API(应用程序接口)在检查前缀或后缀时避免对字符串进行切片。用startswith()和endswith()代替，因为它们是明确的并且错误更少。例如：

No:if foo[:3] == 'bar':
Yes:if foo.startswith('bar'):

例外是如果你的代码必须工作在Python 1.5.2 (但是我们希望它不会发生！)，对象类型的比较应该始终用isinstance()代替直接比较类型，例如：

No:if type(obj) is type(1):
Yes:if isinstance(obj, int):

检查一个对象是否是字符串时，紧记它也可能是unicode字符串！在Python 2.3，str和unicode有公共的基类，basestring，所以你可以这样做：

if isinstance(obj,basestring):

在Python 2.2类型模块为此定义了StringTypes类型，例如：

#!Python
from types import StringTypes
if isinstance(obj, StringTypes):

在Python 2.0和2.1，你应该这样做：

#!Python
from types import StringType, UnicodeType
if isinstance(obj, StringType) or \
isinstance(obj,UnicodeType):

对序列，(字符串，列表，元组)，使用空列表是false这个事实，因此if not seq或if seq比if len(seq)或if not len(seq)好。书写字符串文字时不要依赖于有意义的后置空格。这种后置空格在视觉上是不可辨别的，并且有些编辑器(特别是近来，reindent.py)会将它们修整掉。不要用==来比较布尔型的值以确定是True或False(布尔型是Pythn 2.3中新增的)

No: if greeting == True:
Yes: if greeting:
No: if greeting == True:
Yes: if greeting:

Display mathematics formula in blog with MathJax

David Dong — Wed, 20 Mar 2019 11:11:12 +0800

When writing posts, I need to display mathematics formulas sometimes. I write posts with Markdown, I don’t want to save the formula into an image and load the image in markdown. It is better to write the Latex formula in the markdown file directly. MathJax helped me. This article will introduce what MathJax is and how to use MathJax.

What is MathJax?

This is the definition of the MathJax in MathJax official website.

MathJax is an open-source JavaScript display engine for LaTeX, MathML, and AsciiMath notation that works in all modern browsers.

In short, MathJax is a simple, yet powerful, way of including Tex/LaTex/MathML based mathematics in HTML webpages.

To me, it brought some benefits when writing a blog.

No need for plugin support in Jekyll. I don’t have to worry that it cannot be used when deploying my website at Github Pages for the reason of security checking. (Github Pages uses the Jekyll --safe flag)
Unlike using bitmaps image to display, it is text editing and transforms the mathematics formula to HTML or SVG, so it is workable for zoom scale and is suitable to display under different size screen.
I can write the mathematics formula in markdown with plain text just by following the syntax of LaTex/TeX.
Simple to use, I can quickly use it even I don’t have much knowledge about it.

Moreover, looks MathJax has become into the most popular solution for rendering the Latex mathematics formula in web applications.

At least, I think so.

How to use?

Including MathJax in a webpage

As I said, one of the benefits of MathJax is that it doesn’t need to install any plugin to use, just introducing a piece of JavaScript code into the webpage’s HTML.

<script type="text/javascript" async src="//cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-MML-AM_CHTML">
</script>

I added the above snippet into my default.html, it can also be placed into the head.html or any page’s HTML where you want to use MathJax.

Configuration

Here TeX-MML-AM_CHTML is the configuration for common user cases.

For more information of configuration, read at here.

MathJax offers a global object named MathJax that contains configuration data for the various components.

For TeX or LaTeX input component, it has two types. One is in-line mathematics(display in paragraph) and the other is displayed mathematics(display as a single paragraph).

It uses the double dollar sign $$ as the default math delimiters for displayed mathematics, uses the (\...\) as the math delimiters for in-line mathematics.

The math delimiters can be customized as well.

Here is an example, I wrote below code to add single dollar $ signs as in-line math delimiters.

<script type="text/x-mathjax-config">
  MathJax.Hub.Config({tex2jax: {inlineMath: [['$','$'], ['\\(','\\)']]}});
</script>

It is better to place the MathJax object in a <script> tag just before the script that loads MathJax itself.

My last code is

<script type="text/x-mathjax-config">
  MathJax.Hub.Config({tex2jax: {inlineMath: [['$','$'], ['\\(','\\)']]}});
</script>
<script type="text/javascript" async src="//cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-MML-AM_CHTML">
</script>

Examples

I wrote the below words as a startup.

Hi, there,
This is my first MathJax equation.
When $a \ne 0$, there are two solutions to

$$ax^2 + bx + c = 0$$

and they are

$$x = {-b \pm \sqrt{b^2-4ac} \over 2a}.$$

Write at the last

I found introducing the MathJax javascript code snippet will slow the website’s loading speed. So I re-located the loading code of MathJax to the post’s markdown file, only load it when the post needs a mathematics expression.
I am not familiar with Latex syntax and I will spend some time learning. It is not a problem! 😛

Python Pillow library usage

David Dong — Thu, 07 Mar 2019 20:59:29 +0800

I recently completed some image processing works by Pillow Library. Pillow provides a powerful function and has friendly APIs to the developer. I thought it is necessary to make some notes here for the usage.

What is Pillow?

We cannot say Pillow is PIL, even total from PIL. Actually, Pillow is an open-source project of Github, it is more like a derived branch of PIL. We all know PIL (Python Image Library) is the previous third-party image processing library of python and is almost regarded as the official image processing library of Python because of the powerful function. However, the last PIL release is at build 1.1.7 in 2009 which can only support Python 2. PIL’s release schedule is too infrequent so that some volunteers created an open-source project on Github for providing a compatible version on the basis of PIL. It’s called Pillow. Pillow added in some features based on PIL and provided support to Python 3. With the update of the version and many bug fixes from many contributors, Pillow has developed into a more dynamic image processing library than PIL itself. Pillow can be said to have replaced PIL.

The source code of Pillow is at Github repository.
The official website is here.

What Can Pillow do?

Pillow provides fairly powerful image processing capabilities for Python. It can do many things related to image processing and support the extensive file format.

Below is the official description of Pillow.

Image Archives The Python Imaging Library is ideal for image archival and batch processing applications. You can use the library to create thumbnails, convert between file formats, print images, etc.
The current version identifies and reads a large number of formats. Write support is intentionally restricted to the most commonly used interchange and presentation formats.
Image Display The current release includes Tk PhotoImage and BitmapImage interfaces, as well as a Windows DIB interface that can be used with PythonWin and other Windows-based toolkits. Many other GUI toolkits come with some kind of PIL support.
For debugging, there’s also a show() method which saves an image to disk, and calls an external display utility.
Image Processing The library contains basic image processing functionality, including point operations, filtering with a set of built-in convolution kernels, and color space conversions.
The library also supports image resizing, rotation, and arbitrary affine transform.
There’s a histogram method allowing you to pull some statistics out of an image. This can be used for automatic contrast enhancement and global statistical analysis.

Usage

Installation

Below is the python version information that Pillow can support.

Pillow has been packaged into a standard Python library so the installation is quite easy, uses pip complete installation.

Here is a reminder that Pillow and PIL cannot co-exist in the same environment. Before installing Pillow, need to uninstall PIL firstly.

Install Pillow with pip:

python3 -m pip install --upgrade pip
python3 -m pip install --upgrade Pillow

Image Class

The most important class in the Python Imaging Library is the Image class.
Before use the library, need to import it first.

#python2 
import Image 

#python3 
from PIL import Image

Some useful methods

Here we take the example of python3.

Open()/Show()
To load an image from a file, use the open() function in the Image module. If successful, this function returns an Image object. You can now use instance attributes to examine the file contents:

from PIL import Image
im = Image.open("python-pillow-02.jpg")
print(im.size,im.mode,im.format)
(750, 500) RGB JPEG
im.show()

The format attribute identifies the source of an image. If the image was not read from a file, it is set to None. The size attribute is a 2-tuple containing width and height (in pixels). The mode attribute defines the number and names of the bands in the image, and also the pixel type and depth. Common modes are L (luminance) for greyscale images, RGB for true color images, and CMYK for pre-press images.
Above four-row codes will open and display an image on the screen. it will use the default image viewer of your computer to show the image.

resize()

print(im.size)
(750, 500)
im = im.resize((150,100))
im.show()
print(im.size)
(150, 100)

transpose()

im_rotate_180 = im.transpose(Image.ROTATE_180)
im_rotate_180.show()

The parameters can be
-Image.FLIP_LEFT_RIGHT, which means to flip the image left and right
-Image.FLIP_TOP_BOTTOM, which means to flip the image up and down
-Image.ROTATE_90, which means to rotate the image 90 ° anticlockwise
-Image.ROTATE_180, which means to rotate the image 180 ° anticlockwise
-Image.ROTATE_270, Means to rotate the image 270 ° anticlockwise
-Image.TRANSPOSE, which means to rotate the image (equivalent to 90 ° clockwise rotation)
-Image.TRANSVERSE, which means to transpose the image and then turn it horizontally

crop()

im = Image.open("python-pillow-02.JPG")
box = (100,100,200,200)
region = im.crop(box)
region.show()

The above code cuts out a box rectangle area on the im and displays it. The box is a tuple with four numbers (upper left, upper left, lower right, lower right). Each pixel represents a coordinate unit. Crop() still returns an image object.

filter()
Image can provide below filters.
BLUR、CONTOUR、DETAIL、EDGE_ENHANCE、EDGE_ENHANCE_MORE、
EMBOSS、FIND_EDGES、SMOOTH、SMOOTH_MORE、SHARPEN.
Below we demonstrate some of the effect of filters.

from PIL import Image
from PIL import ImageFilter
im = Image.open("python-pillow-02.JPG")
im = im.resize((150,100))
blur = im.filter(ImageFilter.BLUR)
conf = im.filter(ImageFilter.CONTOUR)
edge = im.filter(ImageFilter.FIND_EDGES)
smooth = im.filter(ImageFilter.SMOOTH)
sharpen = im.filter(ImageFilter.SHARPEN)
emboss = im.filter(ImageFilter.EMBOSS)
blur.show()
conf.show()
edge.show()
smooth.show()
sharpen.show()
emboss.show()

BLUR

CONTOUR

EDGE

SMOOTH

SHARPEN

EMBOSS

rotate()

from PIL import Image
im = Image.open("python-pillow-02.JPG")
w,h = im.size
im = im.resize(((int)(w/2),(int)(h/2)))
im_45 = im.rotate(45)
im_90 = im.rotate(90)
im_45.show()
im_90.show()

rotate 45 degree

rotate 90 degree

convert()
Convert() is a method of image instance object. It accepts a mode parameter to specify a color mode. The value of mode can be as follows:
· 1 (1-bit pixels, black and white, stored with one pixel per byte)
· L (8-bit pixels, black and white)
· P (8-bit pixels, mapped to any other mode using a color palette)
· RGB (3x8-bit pixels, true color)
· RGBA (4x8-bit pixels, true color with transparency mask)
· CMYK (4x8-bit pixels, color separation)
· YCbCr (3x8-bit pixels, color video format)
· I (32-bit signed integer pixels)
· F (32-bit floating point pixels)

from PIL import Image
im = Image.open("python-pillow-02.JPG")
w,h = im.size
im = im.resize(((int)(w/2),(int)(h/2)))
im_1 = im.convert("1")
im_L = im.convert("L")
im_P = im.convert("P")
im_CMYK = im.convert("CMYK")
im_F = im.convert("F")
im_1.show()
im_L.show()
im_P.show()
im_CMYK.show()
im_F.show()

mode 1

mode L

mode p

mode CMYK

mode F

paste()
Paste one image onto another.

from PIL import Image
im = Image.open("python-pillow-02.JPG")
w,h = im.size
im = im.resize(((int)(w/2),(int)(h/2)))
print(im.size)
box = (0,0,100,100)
region = im.crop(box)
im.paste(region,(100,100))
im.show()

in above code, region is the object to be pasted.

split()
Split() method can separate each channel of the original image. For example, for RGB image, the three color channels R, G and B can be separated.

from PIL import Image
im = Image.open("python-pillow-02.JPG")
w,h = im.size
im = im.resize(((int)(w/2),(int)(h/2)))
r,g,b = im.split()
r.show()
g.show()
b.show()

R

G

B

merge()
Merge method and split method are relative. it combines multiple single channel sequences to form a multi-channel image. Mode is the mode of the combined image, such as RGB, and channels is the sequence composed of multiple single channels.

im_merge = Image.merge("RGB",[r,g,b])
im_merge.show()

RawPosts - A theme for Jekyll

David Dong — Sat, 23 Feb 2019 12:21:47 +0800

RawPosts is a simple, elegant, and clean Jekyll theme. It is born from the creation of my blog website. It is designed with a fully responsive feature and can be deployed on Github Pages. I open-sourced this theme on Github with an MIT license, so it is free for use.

After a while using WordPress, I just realized that it was so much bloated. So, I decided to work on my layout, and then, RawPosts was born. It is two columns layout with portfolio-style, I tried to make this theme with minimal size and easy to customize. It is suitable for creating blog website or is built as a foundation for building more meaningful themes.

The latest release is v2.1, I will keep developing this theme. I’ve pinned this post, if any update is available I will update it here.

Preview

Features List

Fully responsive, optimized for tablets & mobiles
Portfolio style
Sidebar with auto TOC generating
Social link (Github,facebook,twitter,linkedin…)
Social media sharing for posts
Markdown and HTML text formatting
Syntax highlighting with Rouge
Pagination of posts
Keywords search
Sticky posts
Archive by date/category
Excerpt management
Related Posts
Comments:
- Gittalk
- Disqus
Statistics:
- Google analytics
- busuanzi
SEO optimization
jemoji for emoji
Light/Dark mode
Chinese/English support
Reading time / post words count statistics
Compatible with GitHub Pages
CI support:
- Travis CI
- Github Actions

Plugins

jekyll-seo-tag
jemoji
kramdown
jekyll-archives
jekyll-paginate
jekyll-toc
rouge
jekyll-tagging-related_posts

Installation

Fork the theme at here.
If you don’t have Jekyll installed, install it on your local machine. If you don’t know how to install, please refer to Getting started with Jekyll (1).
Run the command bundle install in the root of the theme’s folder to install the theme and its dependencies.
Run bundle exec jekyll server to build and serve your site.
The theme is installed successfully here, next you can customize your own website through the _config.yml

Customization & Configuration

You can use the _config.yml file to configure the theme with your preferences.

site personal settings: You’ll need to change the description, title and url to match with your personal information. You’ll also need to replace the logo, default social link and default offline images in the /assets/ directory with your owns.The email needs to be changed to the email you want to receive contact form enquirers with. The default configuration of this theme is my personal information and just replace with yours.
show_excerpts: set to true to show excerpts on the homepage.
paginate: set the number of posts of each pages.
paginate_path: set the path of pages in your site.
sitemap: set to true to generate sitemap.xml content.
dark_mode: set to true to add dark mode toggle.
reading_time: set to true to add reading time statistics.
archives: set to true to generate archives page.
categories: set to true to generate categories page.
gittalk: set to true to add gittalk.
show_statistics: set to true to show statistics of site visitors number.
rss: set to true to add rss.
source_code: set to true to add link to source code.

Include Files

There are some necessary .html files for utilizing the site’s features and they are in different folders.

index.html: index.html is the entry of homepage and is in the /blog folder.
archive.html: archive.html is for the archives function and is in the /archive folder.
category.html: category.html is used for category of the posts and is in the /category folder.
about.html: For about page contents display and is in the /about folder.
pagination.html: For paginate the pages and is in the _includes folder.
reading_time.html: reading_time.html is used for statistic the reading time of posts and display. It is in _includes folder.
menu.html: Used for setting the menu bar of the homepage, is in the _includes folder.

Development

To set up your environment to develop this theme:

Clone this repo
cd into the root directory of your repo and run bundle install.

To test the theme locally as you make changes to it:

cd into the root directory of the repo (e.g. jekyll-theme-rawposts).
Run jekyll server to preview and open your browser to http://localhost:4000/your_baserul/.

This starts a Jekyll server using the theme’s files and contents of the / directory. As modifications are made, refresh your browser to see any changes.

Pull Requests

When submitting a pull request:

Clone the repo.
Create a branch off of master and give it a meaningful name (e.g. my-awesome-new-feature) and describe the feature or fix.
Open a pull request on GitHub.

Welcome to submitting pull requests to me, for each request, I will review as soon as possible and merge any good submits.

Version

2.1.0

Q & A

Welcome raise issues if you have any questions about this theme, not limited for any usage, bug fix, new features requirements… :smile:

License

The theme is available as open source under the terms of the MIT License.

中文 -> 关于本博客模板使用上的一些问题 | David Dong’s Blog

关于本博客模板使用上的一些问题

David Dong — Sat, 23 Feb 2019 12:21:47 +0800

本博客使用了 RawPosts 主题模板，不时有网友在 fork 使用该主题的时候提出了一些问题，这里我将该主题的使用方法汇总成一篇帖子，方便大家参考。

1.关于该主题

RawPosts 是一款轻量级,风格简洁清新的 Jekyll 主题。该主题配置简单并支持丰富的特性,你可以将它用于Jekeyll网站或者个人博客的构建。该主题可以被用于部署在Github Pages上。

预览

2.功能列表

响应式设计, 针对平板 & 手机屏幕设计优化
portfolio 主题图片设置
个性化社交链接（支持电子邮件，Github，Facebook，Twitter，领英等）
社交媒体分享（Linkedin, Facebook, Twitter, Weixin…,180+）
中英文双语支持
分页功能
侧边栏集成文章目录自动生成
文章归档
文章分类索引
摘要管理
全站关键字搜索
Rouge 语法高亮
相似文章归类索引
评论模块：
- Gittalk
- Disqus
  （中国大陆用户推荐使用 Gittalk).
文章阅读量统计：
- busuanzi
- google analytics
站点访问统计：
- busuanzi
- google analytics
  （中国大陆用户推荐使用 busuanzi).
SEO 优化
自动构建集成：
- Travis CI
- Github Actions
jemoji emoji表情支持
两种皮肤可切换：浅色/深色
文章字数统计
阅读时间统计
文章置顶

3.应用插件

4.主题安装

fork 主题到你的 Github 仓库。
复制该主题的仓库到本地。
如果你还没有安装 Jekyll，请先安装 Jekyll。如果你不知道如何安装 Jekyll，可以参考 Getting started with Jekyll (1)。
在本地项目的根目录下运行命令 bundle install 来安装主题以及该主题的依赖。
运行 bundle exec jekyll server 构建项目并生成网站。
至此安装完成! :v: 下一步你需要通过配置 _config.yml来定制你自己的网站。

5.客制化 & 配置

你可以通过配置 _config.yml 文件来客制化该主题。

5.1 客制化设定

你需要更改描述、标题和url以便与个人信息匹配。你还需要将/assets/目录中的 logo、默认社交链接和默认avatar图像替换为您自己的图片，并需要将电子邮件更改为您要接收的联系人表单查询的电子邮件。当前此主题的模板中的默认配置为我个人的信息，将该部分替换为你自己的信息。

show_excerpts: _config.yml中设置该字段为 true 将会启用目录的摘要功能。
search: _config.yml中设置该字段为 true 将会启用全站关键字搜索功能。
paginate: _config.yml中通过设置该字段来设定每页最大的文章数量。
paginate_path: _config.yml中设置该字段为你需要放置子页的路径。
sitemap: _config.yml中设置该字段为 true 将生成 sitemap.html 文件。
syntax: _config.yml中设置该字段，可以选择代码高亮的主题，目前支持的主题有23种。详见 Rouge Theme
related_posts: _config.yml中设置该字段为 true 将会启用相似文章归类索引。
dark_mode: _config.yml中设置该字段为 true 将会启用网站的深色模式切换功能。
reading_time: _config.yml中设置该字段为 true 将会增加文章的阅读时间和字数统计并显示。
archives: _config.yml中设置该字段为 true 将会启用文章的时间归档功能。
categories: _config.yml中设置该字段为 true 将会启用文章的归类功能。
analytics_id: _config.yml中设置该字段为 Google Analytics 的用户 ID，如果填写正确则启用 Google Analytics 分析。
gittalk: _config.yml中设置该字段为 true 将会增加gittalk评论模块。
show_statistics: _config.yml中设置该字段为 true 将会启用网站访客统计功能，默认busuanzi引擎。
rss: _config.yml中设置该字段为 true 将会启用rss生成功能。
source_code: _config.yml中设置该字段为 true 将会增加源代码索引。
share_media: _config.yml中设置该字段为 true 将会启用文章的社交媒体分析，默认媒体有 Twitter/Facebook/微信/linkin。

5.2 包含文件

该主题有一些必要的 .html 文件来实现网站的功能，它们位于不同的文件夹中。

index.html: index.html 是主页的入口文件，生成在项目根目录 /blog 文件夹中。
archive.html: archive.html 实现文章的归档功能，生成在根目录 /archive 文件夹中。
category.html: category.html 实现文章的分类功能，生成在根目录 /category 文件夹中。
about.html: about.html 文件内容对应网站的「关于」页面，里面的内容多为个人相关，将它们替换成你自己的信息，存在于根目录 /about 文件夹中。
pagination.html: pagination.html 实现网站的分页功能，存放在根目录 _includes 文件夹中。
reading_time.html: reading_time.html 统计文章的字数和阅读时间并显示. 存放在根目录 _includes 文件夹中。
menu.html: 生成主页的功能菜单项, 存放在根目录 _includes 文件夹中。
toc.html: 自动生成文章的目录, 存放在根目录 _includes 文件夹中。

6.开发

6.1 搭建开发环境

以下用于配置该主题的开发环境:

复制该主题的仓库到本地；
进入到该主题项目的根目录并执行bundle install。

在你对该主题做了修改后可以在本地测试然后再推送到远端:

进入到该主题本地的根目录 (比如 jekyll-theme-rawposts)；
运行 jekyll server 构建该项目并生成网站，在浏览器中打开http://localhost:4000/你设定的baseurl 来预览网站。再度修改后可以通过刷新网页来预览效果。

6.2 Pull Requests

当你需要提交一个 pull request时，可以：

复制该项目；
创建一个分支并对该分支起一个容易辨认的名字并填写好问题或者需求描述，更改记录；
在Github上提交该分支到一个 pull request。

我平时的编程主要应用在移动端和嵌入式设备上面，偶尔写写前端，纯属个人爱好。因此该主题模板目前看写的还比较粗糙，需要持续改进。欢迎大家提 pull requests 给我, 对于每一个pull request, 我都会尽快review，对于好的建议我都会merge进该主题的代码中，包括但不限于提交问题或需求，修复代码等。

7.其他信息

7.1 版本

2.1.0

7.2 协议

该主题为开源软件，基于MIT协议。

8.问题和回答 (Q&A)

如果你有针对该主题的任何问题，欢迎在下面评论区留言，每个问题我都会认真回复。:smile:

Install python3.6 on ubuntu16.04

David Dong — Wed, 23 Jan 2019 23:07:49 +0800

Today I spent some time installing the python3.6 on my ubuntu16.04. Here is a short note for the process, which might be useful for anyone who wants to do the same thing.

The first thing before you start to install the python3.6 is that you need to know which version of python is on your system. On ubuntu16.04 you can use the below command to check your python version.

ls /usr/local/lib/

For example, before the installation, I have python2.7 and python3.5 on my ubuntu system. They are actually default installed by ubuntu16.04 build.

david@david-VirtualBox:/$ ls /usr/local/lib/
python2.7  python3.5  
david@david-VirtualBox:/$

then input python --version to check which version python is your default python application.

python --version

If you want to switch to another python version, input the below command

echo alias python=python3 >> ~/.bashrc
source ~/.bashrc
python --version

Here I switched my default python version to python3.5. But python3.5 is still not the right one, what I need is python3.6. To install python3.6, there are usually two methods. one is using apt-get install. The command is as below.

sudo add-apt-repository ppa:jonathonf/python-3.6
sudo apt-get update
sudo apt-get install python3.6

Unfortunately, it doesn’t work on my installation for the PPA has been removed. Therefore I turned to the second way to download the source code and compile, install it. The command as below.

wget https://www.python.org/ftp/python/3.6.0/Python-3.6.0.tar.xz
xz -d Python-3.6.0.tar.xz
tar -xvf  Python-3.6.0.tar
cd Python-3.6.0
./configure
make
sudo make install

Waiting for the installation to complete. Then check the python version, if you see the “Python 3.6.x”, python3.6 has been installed successfully on your ubuntu16.04.

david@david-VirtualBox:~$ python --version
Python 3.6.0

If you don’t see the Python3.6.x, instead, you see Python3.5.x or else, which means your system’s default python version isn’t python3.6. You need to use update-alternatives --config python command to switch it to python3.6.
Like this.

sudo update-alternatives --list python
sudo update-alternatives --config python

If you find below message, represents the alternatives failed to recognize python and you must install python into your alternatives list firstly.

update-alternatives：error：no alternatives for python

install your python by below command.

sudo update-alternatives --install /usr/bin/python python /home/david/Python-3.6.0 3
sudo update-alternatives --install /usr/bin/python python /usr/bin/python3.5 2
sudo update-alternatives --install /usr/bin/python python /usr/bin/python2.7 1

The last character of above each command sets the priority of your pythons on the system. The bigger the higher priority. After installment, check again. you will see all your pythons are in the alternatives.

david@david-VirtualBox:~$ sudo update-alternatives --list python
/home/david/Python-3.6.0
/usr/bin/python2.7
/usr/bin/python3.5

Then switch to root user and execute the last command to choose which python you are going to set as default.

sudo su
update-alternatives --config python

If you see the below menu, congratulations, just select the one you want.

有 3 个候选项可用于替换 python (提供 /usr/bin/python)。

  选择       路径                    优先级  状态
------------------------------------------------------------
* 0            /home/david/Python-3.6.0   3         自动模式
  1            /home/david/Python-3.6.0   3         手动模式
  2            /usr/bin/python2.7         1         手动模式
  3            /usr/bin/python3.5         2         手动模式

要维持当前值[*]请按<回车键>，或者键入选择的编号：

We have done all the job for now. Hope this short article can give you help if you have the same requirement as me. If you have any questions, please ask in below comment box.

Use jekyll-toc plugin on Github Pages

David Dong — Sat, 29 Dec 2018 21:11:16 +0800

Today I want to add an auto TOC (Table of Contents) function to my blog. I used to write TOC manually and it is really boring. I found a good plugin jekyll-toc in Github, which can fully meet my requirement.

Start

The installation is simple, I forked and installed the plugin successfully in 5 minutes.

After then, I was going to separate the TOC and post contents. I designed a sidebar to show the TOC independently by placing the {% toc %} tag into the sidebar.

<!-- put sidebar! -->
<div class="post-index-container">
	{% if page.language == 'chinese' %}
		<li class = "post-index">文章目录</li>
	{% else %}
		<li class = "post-index">On this page</li>
	{% endif %}
		<li>{% toc %}</li>
</div>

I built the code locally by Jekyll and reloaded the page.

Cheers!

The TOC appeared, this was exactly what I want!

I got a trouble

Things are going well so far, it is so easy?

No, I got into trouble when I committed the change to the Github repository. The site was built failed by the Github Pages Generator.

The tag `toc` on line 32 in `/_layouts/post.html` is not a recognized Liquid tag.

I realized immediately that this plugin wasn’t in the whitelist of Github Pages. I forgot that!

I have to use a trick way to skip the restriction. As the above bulletin showed, I need to build the site locally and push the generated static files to my Github repository.

Don't use Github Pages Generator build.

Use Master branch

Followed the above information I checked the root path of my blog project in Github, it was master branch.

So I added the site file _site/ to the master branch and pushed it to the origin repository.

I referred to two articles¹ here.

git add _site/*
git commit -m "add static site file"
git push origin master

However, it didn’t work!

Use gh-pages branch

There should be something wrong with the above procedure.

I went back to google to search for another solution.

Thanks to these two articles².

I made it clear when I build the website by Github Pages Generator, it actually creates a new branch gh-pages under master branch, where it stores the static site files for access. Therefore I need to create the gh-pages branch and place the generated static HTML there.

I changed the command as below.

cp -r _site /tmp/
git checkout -b gh-pages
rm -rf *
cp -r /tmp/_site/* ./
git add -A
git commit -m "add site files"
git push origin gh-pages

It worked this time!

Conclusion

If you want to deploy your website on Github Pages with the unsupported plugins, you will need to build your site locally and create gh-pages branch and upload your site file to this branch. One important point is that you should use the branch as the root, which means placing the site file into the directory directly instead of a sub-directory.

Syntax Highlighting your code in Jekyll with Rouge

David Dong — Sat, 22 Dec 2018 23:09:12 +0800

Syntax highlighting is important (especially for programming blogs) because it improves the readability of posts. This article will introduce a syntax highlighter - Rouge.

I think maybe someone who attaches a code snippet into the blog wants to get a beautiful syntax highlighting style, just like we see in some popular IDEs, Visual Code, Eceplise, Atom…

I also like that, so I decided to add a syntax highlighter to my blog.

My blog was constructed by Jekyll, Jekyll also has built-in support for syntax highlighting of code snippets using either Rouge or Pygments, and Rouge is the default syntax highlighter in Jekyll 3 and above. It’s also supported by Github Pages, which means the website that using Rouge can be deployed on Github Pages.

Install Rouge

There is nothing special about the installation, like an ordinary Jekyll plugin installation.

gem install rouge

and declare in the _config.yml

highlighter: rouge

If you use kramdown, add this

markdown: kramdown
kramdown:
input: GFM
syntax_highlighter: rouge

Syntax highlighting with Rouge

Rouge can support syntax highlighting of over 100 languages, find them here.

To render a code block with syntax highlighting, surround your code as

{% highlight ruby %}
def foo
  puts 'foo'
end
{% endhighlight %}

Including the linenos argument will force the highlighted code to include line numbers.

Stylesheet

There are some syntax highlighting style themes available in Rouge, you can look for them at

Rouge Themes Doc / Rouge themes

Rouge comes built-in with rougify, a command-line tool that converts a style theme to a CSS file.

Use below command to view the themes that Rouge supports.

rougify help style

As of Rouge 1.11.1, the available themes are:

base16, 
base16.dark, 
base16.monokai, 
base16.monokai.light, 
base16.solarized, 
base16.solarized.dark, 
colorful, 
github, 
gruvbox, 
gruvbox.light, 
molokai, 
monokai, 
monokai.sublime, 
thankful_eyes

Use below command to generate CSS file for the syntax highlighting style you want.

rougify style monokai.sublime > syntax_monokai.css

Copy the generated style CSS file to your site’s style folder and don’t forget to include the stylesheet file into your head.html.

<link href="https://cdn.jsdelivr.net/gh/gangdong/gangdong.github.io@dev/assets/css/syntax_monokai.css" rel="stylesheet"/>

And that’s all you need to start having syntax highlighting on your Jekyll site using Rouge.

Hope that’s somehow useful for you! 🙂

How did I implement related posts function in my blog

David Dong — Wed, 14 Nov 2018 10:51:56 +0800

What will you learn in this post?
This post introduced how to implement the related posts function with Jekyll.

Jekyll’s Native Support

My blog was constructed with Jekyll, Jekyll has native support for related posts function, which can be used with Liquid as site.related_posts.

However, when I used it, I found this function has some points not as good as I expected.

problem

This function collects the related posts and group them by searching the same tags among them. But if there aren’t enough posts sharing the same tags. It will return ten most recent posts as default.

This behavior doesn’t make sense to me!

Why should a post be related to another post that they have nothing to do with the contents just because they are close in release time and no other related posts are existing?
This function lists posts with mutual tags without ordering by how many mutual tags they share. In this way, if A post is tagged with a, b and c, B post is tagged with a and C post is tagged with a and c, There is no problem if you do not limit the count of the related posts. It will return B and C as A’s related posts. But if you set the limit to 1, A’s related posts may contain only B instead of C. Apparently, C should be the right one under this case since it is more related to A.
To achieve this function, need to run the Jekyll command with the --lsi (latent semantic indexing) option when building.

Unfortunately, GitHub Pages does not support the lsi option when generating sites.

jekyll server --lsi

Use plugins

So I decided not to use this function and turned to find other support.

I found two plugins that can support the related posts function.

By reading the README file, both of them have the same feature that overrides the built-in related_posts function to calculate related posts based on a posts’ tag, and the algorithm is identical.

So I chose one of them to try the function.

I installed the plugin and inserted the below code to my post.html.

{% if site.related_posts.size >= 1 %}
<div>
  <h3>Related Posts</h3>
  <ul>
  {% for related_post in site.related_posts limit: 5 %}
    <li><a href="{{ related_post.url }}">{{ related_post.title }}</a></li>
  {% endfor %}
  </ul>
</div>
{% endif %}

I tested and the above problem was fixed!

The plugin will return nothing if no related posts are found instead of recent posts and look also adding the ordering of the searched tags count.

Only one problem, the Github Pages cannot support these two plugins since they are not in the whitelist and will be blocked to build for security reasons.

Liquid grammar

David Dong — Wed, 03 Oct 2018 23:33:19 +0800

I recently used Liquid language to design my blog website. The Liquid is an open-source template language written in Ruby. This article is a short study note of Liquid and presented some main knowledge.

My blog website was constructed by Jekyll, Jekyll uses Liquid as its template language and adds many objects, tags, and filters. The new content includes objects representing content pages, tags to introduce content fragments into the page, and filters to manipulate strings and URLs.

I do not know this language previously, so I spent some time learning it.

The Liquid code consists of objects, tags, and filters.

objects

Object tells Liquid where to display content on the page. Object and variable names are identified by double curly braces: {{ }}

{{post.name}}

filters

The filter changes the output of the Liquid object. They are used for output, separated by a | symbol.

The keywords include :

abs
append
at_least 
at_most 
capitalize 
ceil
compact 
concat 
date
default 
divided_by
downcase
escape
escape_once
first
floor
join
last
lstrip 
map
minus 
modulo
newline_to_br 
plus
prepend
remove
remove_first 
replace
replace_first
reverse
round
rstrip
size
slice
sort
sort_natural 
split
strip
strip_html
strip_newlines
times
truncate
truncatewords 
uniq
upcase
url_decode
url_encode

Operator

Except the 3 basic elements, Liquid contains the logical and comparison operators.
They are

"=",">","<",">=","<=","!=","or","and".

The syntax is similar to other languages, like C.

Object Type

There are five types of Liquid objects.

String
Number
Boolean
Nil
Array

Actually, except Nil (Nil type represents a null object.), others you can find the same type in other languages (for example C), the usage is quite similar.

Liquid in Jekyll

In Jekyll, Jekyll adds a few handy filters and tags of its own.

Below are some of useful tags and filters, which I have used on my blog website.

filters

relative_url
date_to_string
group_by
where
number_of_words

Calculate reading time in Jekyll

David Dong — Mon, 13 Aug 2018 19:12:43 +0800

I believe it is a good idea to estimate the reading time of every post and display it to readers following the post excerpts.

There are plenty of Jekyll plugins that support realizing this feature, but I don’t suggest using them if you want to deploy your blog on Github Pages.

GitHub Pages may block the working of these plugins by running with the --safe flag

So I tried to write the below trick code to realize this feature.

{% assign words = post.content | strip_html | strip_newlines | remove: " " | size %}
{% assign time = post.content | strip_html | strip_newlines | remove: " " | size 
| divided_by: 400 | plus: 1 %}

<footer>
  <div style="margin-top:25px">
	<a class="post-link" href="{{ post.url | relative_url }}">
		<span class="icon-clock morebox" >Reading time: {{ time }} 
        {%if time > 1 %} mins
        {% else %}min{% endif %}, {{words}} words.<br>      Continue read...</span>
    </a>
  </div>
</footer>

The method is straightforward,

Statistics the words count of every post. Actually a more simple and pure Liquid code is like this:

{% assign words = content | number_of_words %}

But Jekyll’s built-in filter number_of_words cannot accurately count the number of Chinese words. If you have Chinese words to calculate, you can use Liquid’s size filter to avoid this problem. In order to make statistics more accurate, it is better to ignore all HTML tags and blank lines before calculation.

Get a value of WPM (word per minute), by google the WPM value, a person can read 300-500 words per minute on a computer monitor. I read fast and I think it can be a bigger value, but I set a median of 400 here for my blog.
The rest work is easy, we only need to calculate the reading time by dividing word count with WPM.

Next, I create a read_time.html in my _includes folder and put all this code into it to well organize them.

I just include the read_time.html in my post layout.

{% include reading_time.html %}

That’s all. If you want to get all code, fork it here.

Archive in Jekyll

David Dong — Sat, 30 Jun 2018 15:44:56 +0800

After setting up the pagination, the next thing I wanted to add to my blog was a proper archive by both date and category rather than just the reverse-chronological list of posts. Although this wasn’t as straightforward as setting up pagination, a bit of searching turned up some sites that helped me get everything running.

Archive by date

Because a list of posts in reverse-chronological order can be readily accessed in Jekyll through site.posts, all that’s needed is to use some clever Liquid markup to process it. I used below code to group my archives by year.

{% comment %}Calucate the post count for each year{% endcomment %}
{% assign count = 1 %}
{% assign year_cnt = "" %}
{% for post in site.posts reversed %}
    {% assign year = post.date | date: '%Y' %}
    {% assign nyear = post.next.date | date: '%Y' %}
    {% if year != nyear %}
        {% assign count = count | append: ', ' %}
        {% assign counts = counts | append: count %}
		{% assign year_cnt = year_cnt | append: year | append: ', ' %}
		{% assign years = years | append: year_cnt %}
		{% assign year_cnt = "" %}
        {% assign count = 1 %}
    {% else %}
        {% assign count = count | plus: 1 %}
    {% endif %}
{% endfor %}

{% comment %}show navigation tag here{% endcomment %}
<div class="archive-label-meta" style ="margin-top:25px">
{% assign i = 0 %}
{% comment %} remove the separator. {% endcomment %}
{% assign counts_year = counts | split: ', ' | reverse %}
{% assign years = years | split: ', ' %}
{% for year in years reversed %}
<ul class="post-categories">       
     <a href = "#{{year}}" style = "font-family: Raleway,Arial,sans-
      serif; font-size:14px">{{ year }} ( {{counts_year[i]}} )</a>
</ul>
{% assign i = i | plus: 1 %}
{% endfor %}
</div>

{% comment %}list post link for each year{% endcomment %}
{% assign counts = counts | split: ', ' | reverse %}
{% assign i = 0 %}
{% for post in site.posts %}
    {% capture year %}{{ post.date | date: '%Y' }}{% endcapture %}
    {% capture nyear %}{{ post.next.date | date: '%Y' }}{% endcapture %}
      {% if year != nyear %}
        <h3 id = "{{year}}" class="archive-title">{{ post.date | date: '%B %Y' }}</h3>
	  {% assign i = i | plus: 1 %}
      {% endif %}
	<div class="archive-meta" style="margin-left:15px;line-height:16px">
	 <div class="post-date" style="margin-right:30px; font-size:16px; 
          width:180px"><i class="icon-calendar"></i>{{ post.date | date:
          "%e %B %Y" }}</div>
     <a class="post-link" href="{{ post.url | relative_url }}">
     <h2 class="archive-link">{{ post.title | escape }}</h2>
     </a>
   </div>
{% endfor %}

I added a group of tags for statistics on the posts count of every year and afforded the navigation to the posts group. So the first loop will calculate the post count of every year and return two arrays. One is for recording the year index and the other gets the count of the year. With these two arrays, I can add the tags in front of the lists grouped by years.

The second loop uses the {% capture %} tag to get the year of each post and the one that follows it before comparing them. If they’re different, the year is inserted before a link to the post while if they’re the same, the post is simply added to the list for that year.

Archive by category

Jekyll provides variables on the front matter for users grouping the posts by category or tags.

The variable site.categories.CATEGORY and site.tags.TAG return the list of all posts in category CATEGORY or tag TAG. What we do is reading the categories field of each article in the post directory and listing the articles under each category.

I used the category attribute to group my posts so I took site.categories.CATEGORY as an example for interpretation.

site.categories returns an array of objects. Every object of this array contains a CATEGORY name in your front matter and all the posts belong to this CATEGORY. For every element, the index[0] stores the CATEGORY name, and index[1] is a list or array we can also consider as which is a collection of posts that are attached to this CATEGORY.

So the method is simple, we can loop the index[1] of every CATEGORY to get every post and group them by the index[0], which is the name of CATEGORY.

Use below code, I grouped the posts by the CATEGORY of theirs. Similar to archive by date, I added the tags for every CATEGORY to statistic the count of them and navigation.

 
{% comment %}calculate the post count for each category{% endcomment %}
{% assign count = "" %}
{% assign counts = "" %}
{% for tag in site.categories reversed %}
  {% for post in tag[1] %}
	{% assign count = count | plus: 1 %}
  {% endfor %}
  {% assign counts = counts | append: count | append: ', ' %}
  {% assign count = "" %}
{% endfor %}

{% comment %}show navigation label here{% endcomment %}
<div class="archive-label-meta" style ="margin-top:25px">
{% assign i = 0 %}
{% assign counts = counts | split: ', ' %}
{% for tag in site.categories reversed  %}
<ul class="post-categories">       
     <a href="#{{tag[0]}}" style = "font-family: Raleway,Arial,sans-
      serif; font-size:14px">{{ tag[0] }} ( {{counts[i]}} )</a>
</ul>
{% assign i = i | plus: 1 %}
{% endfor %}
</div>

{% comment %}list post link for each category{% endcomment %}
{% for tag in site.categories reversed %}
<div>
<h3 id = "{{tag[0]}}" class="archive-title">{{tag[0]}}</h3>
{% for post in tag[1] %}
{%- assign date_format = site.minimal.date_format | default: "%b %-d, %Y" -%}
    <div class="category-meta" style="margin-left:15px; line-height: 16px">
      <a class="post-link" href="{{ post.url | relative_url }}">
		<h2 class="archive-link" >&bull; {{ post.title | escape }}</h2>
      </a>
        <div class="post-date">
          <span class="icon-calendar" style = "font-size:16px">
            {{ post.date | date: date_format }}</span>
		</div>
    </div>
{% endfor %}
</div>
{% endfor %}

The last work is to create archive.html and category.html files to well organize the codes and display them with the desired layout, add a hyperlink to the HTML files on the website homepage.

That’s about it! Hopefully, this is a useful reference for anyone that’s looking to make their Jekyll archives a bit more accessible. The code is in my github repository, if you are interesting, you can fork it. 🙂

Pagination in Jekyll

David Dong — Thu, 28 Jun 2018 19:43:17 +0800

If you don’t have pagination for your blog and you want to do it, this page is probably helpful to you.

My blog is constructed by Jekyll, so I will only write down how to paginate the blog powered by Jekyll. Actually below part are some experiences when I paginated my blog.

When you have a long post list, breaking them into smaller lists and display them over multiple pages will become a common requirement. Fortunately, Jekyll offers a pagination plugin, which can automatically generate the appropriate files and folders you need for paginated listings.

Install plugin

Add the jekyll-paginate into your website’s Gemfile and declare it at your configuration file _config.yml under plugins.

Like below Gemfile.

gemspec
group :jekyll_plugins do
    gem 'jekyll-feed'
    gem 'jekyll-sitemap'
    gem 'jekyll-paginate'
end

Enable pagination

Add a line to the _config.yml file that specifies how many items should be displayed per page:

paginate: 12

The number should be the maximum number of posts you’d like to be displayed per page in the generated site.

Because Jekyll can only support pagination for HTML files so far, do not work from within Markdown files from your Jekyll site. Pagination works when called from the HTML file, named index.html, which optionally may reside in and produce pagination from within a subdirectory, via the paginate_path configuration value.
You can specify the destination of the pagination pages:

paginate_path: "/blog/page:num/"

Jekyll will search the /blog/ directory and read in blog/index.html, send it to each pagination page, and write the output to blog/page:num/, where :num is the pagination page number, starting with 2.

For example, if you output 3 pages, if you look at the _site directory, you will find a /blog folder with two subfolders /page2 and /page3 in it. Each folder has an index.html file, which contains the contents that need to be displayed.

because the pages start with 2, which means no page1 exists.That will require special handling for the first page when rendering the pages. Actually, page1 contents are displayed on the index.html of /blog directory.

For me, I tried to set the path as below firstly, but failed to load the index.html. After I change to /blog/page:num, it started to work. Who can tell what the reason is here?

paginate_path: "/page:num/"

Attributes

The pagination plugin exposes the paginator Liquid object.
You can find the attributes in my another blog Getting started with Jekyll (2)

Render the pages

You have enabled the pagination so far, next thing is to display your posts in a list using the paginator variable that will now be available to you. Below code is an example from my blog’s pagination.html file. These pieces of code renders a list of each page with links to all but the current page.

<!-- Pagination links -->
{% if paginator.total_pages > 1 %}
<div class="pagination">
  {% if paginator.previous_page %}
    <a href="{{ paginator.previous_page_path | prepend: site.baseurl | replace: '//', '/' }}">&laquo; Prev</a>
  {% else %}
    <span>&laquo; Prev</span>
  {% endif %}

  {% for page in (1..paginator.total_pages) %}
    {% if page == paginator.page %}
      <span class="active">{{ page }}</span>
    {% elsif page == 1 %}
      <a href="{{ '/blog/index.html' | prepend: site.baseurl | replace: '//', '/' }}">{{ page }}</a>
    {% else %}
      <a href="{{ site.paginate_path | prepend: site.baseurl | replace: '//', '/' | replace: ':num', page }}">{{ page }}</a>
    {% endif %}
{% endfor %}
  
  {% if paginator.next_page %}
    <a href="{{ paginator.next_page_path | prepend: site.baseurl | replace: '//', '/' }}">Next &raquo;</a>
  {% else %}
    <span>Next &raquo;</span>
  {% endif %}
</div>
{% endif %}

Besides, you still need to change your page.html or home.html where displays the every pages.
I moved these code to /blog/index.html from page.html and change the loop from site.posts to paginator.posts.
Below code loops through the paginated posts.

    {%- for post in paginator.posts -%}
      <li>
        {%- assign date_format = site.minimal.date_format | default: "%b %-d, %Y" -%}
        <a class="post-link" href="{{ post.url | relative_url }}">
          <h2 class="post-title">{{ post.title | escape }}</h2>
        </a>
        <div class="post-meta">
          <ul class="post-categories">
            {%- for tag in post.categories -%}
              <li>{{ tag }}</li>
            {%- endfor -%}
          </ul>
          <div class="post-date">
            <i class="icon-calendar"></i>
            {{ post.date | date: date_format }}</div>
        </div>
        <div class="post">
          {%- if site.show_excerpts -%}
            {{ post.excerpt }}
          {%- endif -%}
        </div>
		<footer>
          <a class="post-link" href="{{ post.url | relative_url }}">
          <h5 class="morebox">Read More...</h5>
        </a>
        </footer>
      </li>
    {%- endfor -%}

Others

Currently jekyll-paginate plugin doesn’t allow paging over groups of posts linked by a common tag or category. The more recent jekyll-paginate-v2 plugin can support the pagination for categories, tags and collections. See the pagination examples in the repository.

This plugin is not supported by GitHub Pages. I haven't tried it!🙂

Gitalk Usage

David Dong — Sat, 12 May 2018 12:22:01 +0800

If you are looking for a quick solution about how to add a comment function on your blog, this article probably can help you.

I have been trying to add a comment function under my blog for the past few days. My blog was built on Github by Jekyll, Jekyll is just a tool for generating static web pages, and does not provide database support, which means, I need help with the third-party software tool.

Of course, disqus is the first choice that you can think out however it is not a good option for china users for some known reasons. After reviewed some popular third-party tools, I decided to use Gitalk to construct my comment function.

Remember Github is all-powerful, you can find everything on it. I found Gitalk on Github. Gitalk is a comment component based on the Github issue and preact. Actually, it is a Github project, designed for Github users and easy to be applied on the Github pages. Gitalk stores the comments into your repository issues and doesn’t need any extra database.

The official Github page link of Gitalk is Here.

It is amazingly easy to use and no need for database support in the background. Moreover, it can support multiple languages includes Chinese.

The installation is simple, just need to refer to the below procedure.

Register a Github application, as a Github application you need to get authorization before using it. The path to open the OAuth Apps page is
- Github homepage->Settings->Developer Settings->OAuth Apps.

You must specify the website domain URL in the Authorization callback URL field. After registration, you will get a Client ID and Client Secret, remember these two numbers, you will need to fill them in the following step.

Please note, the user number is zero at this moment when you haven’t authenticated the application. You need to do it later. Now you have registered Github application for Gitalk.

Next, copy the below code snippet into the page where you want to use Gitalk or add it to the post’s page, it depends on your website’s structure.

Below are the setting options.

clientID - Github Application Client ID.
type: String.
Required.
clientSecret - Github Application Client Secret.
type: String
Required.
repo - Github repository.
type: String
Required.
owner - Github repository owner. Can be personal user or organization.
type: String
Required.
admin - Github repository owner and collaborators. (Users who having write access to this repository)
type: Array
Required.
id - The unique id of the page. Length must less than 50.
type: String
Default: location.href.
number - The issue ID of the page, if the number attribute is not defined, issue will be located using id.
type: Number
Default: -1.
labels - Github issue labels. Array
type: Array
Default: [‘Gitalk’].
title - Github issue title.
type: String
Default: document.title.
body - Github issue body.
type: String
Default: location.href + header.meta[description].
language - Localization language key, en, zh-CN and zh-TW are currently available.
type: String
Default: navigator.language || navigator.userLanguage.
perPage - Pagination size, with maximum 100.
type: Number
Default: 10.
distractionFreeMode - Facebook-like distraction free mode. Boolean
type: Boolean
Default: false.
pagerDirection - Comment sorting direction, available values are last and first.
type: String
Default: ‘last’
createIssueManually - By default, Gitalk will create a corresponding Github issue for your every single page automatically when the logined user is belong to the admin users. You can create it manually by setting this option to true.
type: Boolean
Default: false.
proxy - Github oauth request reverse proxy for CORS. Why need this?
type: String
Default: https://cors-anywhere.herokuapp.com/ https://github.com/login/oauth/access_token.
flipMoveOptions - Comment list animation. Reference
type: Object
Default:

{
      staggerDelayBy: 150,
      appearAnimation: 'accordionVertical',
      enterAnimation: 'accordionVertical',
      leaveAnimation: 'accordionVertical',
}

enableHotKey - Enable hot key (cmd|ctrl + enter) submit comment.
type: Boolean
Default: true.

Since Gitalk uses the repository issues to store the comments, so don’t forget to enable your repository issue function.

The default setting is enabled.

For now, the Gitalk comment component has been added into your website successfully.
The appearance is like this.

When first loading, you will be requested to sign in your Github account to active the comment function.

Active it then you can use it now!
If you like this page or you have anything would like to share, comment below! 🙂

Mark一下 Markdown 语言的基本语法

David Dong — Wed, 25 Apr 2018 11:03:36 +0800

最近在写博客的时候用到了 Markdown 语言，做为一个学习总结，有必要把常用的语法mark一下。

Markdown 是一种轻量级的「标记语言」，它允许人们使用易读易写的纯文本格式编写文档。

Markdown 的语法十分简单。常用的标记符号也不超过十个，Markdown 编写的文档可以导出 HTML 、Word、图像、PDF、Epub 等多种格式的文档。

Markdown 的语法学习归纳起来可以从下面几个方面来学习。

字体 2. 标题 3. 图片 4. 链接 5. 分割线 6. 列表 7. 表格 8. 代码

字体

语法

斜体：文字两边用 * 包起来
粗体：文字两边用 ** 包起来
粗斜体：文字两边用 *** 包起来
删除线：文字两边用 ~~ 包起来

示列如下：

   *斜体*
   **粗体**
   ***粗斜体***
   ~~删除线~~

效果如下：
斜体
粗体
粗斜体
~~删除线~~

标题

语法
共分为6级标题，分别在前面加 # ，同时在#号后加一个空格。示列：

# 一级标题
## 二级标题
### 三级标题
#### 四级标题

效果如下：

一级标题

二级标题

三级标题

四级标题

依次论推…

图片

语法

![picname](图片地址''图片title'')

效果如下：

链接

语法
和图片的语法很类似，只是前面缺少一个!。

[名称](地址''title'')

效果如下：
DAVID DONG的博客

分割线

语法
三个或者更多的 --- 或者 ***

我是分割线 
---

效果如下:

我是分割线

列表

无序列表

语法
在内容前使用* - + 都可以。

- 名称
- 年龄
- 性别

效果如下:

名称
年龄
性别

有序列表

语法
在内容前使用数字。

1. 名称
2. 年龄
3. 性别

效果如下:

名称
年龄
性别

表格

语法
语法如下，其中第二行的 ---: 表示了对齐方式，默认左对齐，还有右对齐（右边加：） 和居中（两边加：）

表头|表头|表头
---|:--:|---:
内容|内容|内容
内容|内容|内容

效果如下:

语言	面向对象	动态语言
Java	是	否
C	否	否
Python	是	是

代码

语法
对于码农来说是非常有用的，方便打印代码在博客里。对于单条语句用'包裹语句即可。对于代码块，用'''包裹即可。
效果如下:

int void main(){
	printf("just show how to print me!");
	return 1;
}

到这里 markdown 一些基本的命令就讲完了，用 markdown 写文档还是很方便的。最后再介绍几款好用的 markdown 的编辑器，方便网友使用。
目前比较主流的支持 markdown 语言的编辑有 markdownpad,typora,sublime,Mou,atom,Cmd Markdown 等，这些软件大部分都能支持 windows/liunx 和 ios(Mou只支持ios) 的平台。而且大部分是免费使用。功能上大同小异，都支持实时预览和 HTML/PDF 输出，有些还能够自定义语法的高亮显示等。我目前使用的是 markdownpad2,编辑起来还是很方便的。

Getting started with Jekyll (2)

David Dong — Fri, 13 Apr 2018 23:21:15 +0800

This page will continue to give a brief introduction of the Jekyll grammar.

After reading last article you should complete the installation of Jekyll and download a Jekyll theme on your computer.

Let’s start this article from the source code of the theme you downloaded.

Jekyll directory structure

Unzip the package you downloaded and check the directory structure. The Jekyll directory structure mainly includes the following directories.

The description of each folder.

Every file or directory beginning with the following characters: ., _ , # or ~ in the source directory will not be included in the destination folder. Such paths will have to be explicitly specified via the config file in the include directive to make sure they’re copied over.

Liquid

Jekyll uses the Liquid templating language to process templates. Generally in Liquid, you output content using two curly braces and perform logic statements by surrounding them in a curly brace percentage sign.

To learn more about Liquid, check out the official Liquid Documentation.

Variables

Jekyll traverses your site looking for files to process. Any files with front matter are subject to processing. For each of these files, Jekyll makes a variety of data available via Liquid. The following is a reference to the available data.

global variables

site variables

page variables

paginator variables

Above content is the description from the official documents.

Talk is always easy, next, I will write some codes to demonstrate the usage.

Below content shows the variables value of my blog.

This page's title is Getting started with Jekyll (2)
This page's name is 2018-04-13-Web-jekyll-grammar.markdown
This page's url is /blog/2018/04/13/Web-jekyll-grammar.html
This page's id is /blog/2018/04/13/Web-jekyll-grammar
This page's date is 2018-04-13 23:21:15 +0800
This page's categories is Blog
This page's collection is posts
This page's tags is Blog
This page's dir is 
This page's path is _posts/2018-04-13-Web-jekyll-grammar.markdown
This site's url is http://localhost:4000

Jekyll command

Jekyll has below commands

Jekyll new PATH: Create a new site in the specified directory using the gem-based default theme.
Jekyll new PATH --blank:Create a new blank site in the specified directory.
Jekyll build or Jekyll b:Perform a build and output the generated site to the. / site (default) directory.
Jekyll serve or Jekyll s:Build site and provide local access service when source file changes.
Jekyll doctor:Output any deprecated features or configuration issues.
Jekyll clean:Delete all generated files: destination folder, metadata file, sass, and Jekyll caches.
Jekyll help:Display help information or display help information for specific subcommands, such as Jekyll help build.
Jekyll new-theme:Creates a new Jekyll theme scaffold.

Above contents are not include all of Jekyll’s grammar, there are some other parts, such as

Jekyll configuration
Includes
Layouts
Permalinks
Themes
Pagination

Due to the blog’s length limitation, I cannot list all of the contents, if you are interested in the other part, please find more details at the below website Jekyll Tutorial.

Getting started with Jekyll (1)

David Dong — Tue, 27 Mar 2018 20:56:30 +0800

In my last article, I have introduced how to create a blog based on Github Pages, if you haven’t read it and you want to read it, find the article at here. Following that article, this page will give a brief introduction to Jekyll and its installation.

Jekyll

What’s Jekyll?

Jekyll is a simple and free blog generation tool, similar to WordPress. But it’s very different from WordPress because Jekyll is just a tool for generating static web pages and does not need database support. It can work with third-party services, such as Disqus. The key is that Jekyll can be deployed on GitHub Pages free of charge and can bind its own domain name.

Github Pages use Jekyll to build and generate the website.

As a static site generator, Jekyll provides many advantages. For example, a website constructed by Jekyll will get a faster page speed over a dynamic, database-driven site.

Jekyll also offers a way of storing data in a more human-readable format, such as YAML or JSON. This means you can store data that is used in multiple places across your site in one file and reference it in many places.

Others are Jekyll gives the developer more freedom to design and build the site. It is free to write your own HTML and text onto the page in the format you want.

Generally, Jekyll is a static site generator. It will transform your plain text into static websites and blogs. No more databases, slow loading websites, risk of being hacked…just your content.

Let’s turn to how to build the website by Jekyll, I suggest you construct your website on Github Pages because Jekyll is well deployed on Github Pages and there are many Jekyll theme templates for you powering your website. More important is all of these are free of charge. If you like, you can even bind your domain on it. you can view my previous article to learn how to create a blog on GitHub Pages.

Installation

The first step is installing the Jekyll tools on your PC, here I just introduce the installation on the windows platform.

download Ruby at here and install it. You can skip it if you have already installed.
download RubyGems at here. Again, skip this step if you have done.
unzip the RubyGems folder and enter the directory. Execute the below command.

ruby setup.rb

execute command install Jekyll.

gem install Jekyll

After the installation, check it by creating a Jekyll template.

cd d:
d:
Jekyll new testblog
cd testblog
Jekyll server

If you see below output message, Jekyll is working on your PC now.

Generating...
    done in 1.684 seconds.
Please add the following to your Gemfile to avoid polling for changes:
    gem 'wdm', '>= 0.1.0' if Gem.win_platform?
Auto-regeneration: enabled for 'D:/Study/backup/daviddong.github.io'
    Server address: http://127.0.0.1:4000
Server running... press ctrl-c to stop.

So far you can input http://127.0.0.1:4000/ in the browser to view the blog you just created.

Jekyll theme

Jekyll provides a lot of beautiful theme templates, you can use them to power your website easily.

You can view the demo, download the source code and use it on your own website.

Now go and choose the one you like -> Jekyll theme.

In next article, I will introduce Jekyll grammar. If you have questions about this article, please add them to the comment box. I will try to answer you as promptly as I can.

How to build your personal blog on Github Pages

David Dong — Fri, 23 Mar 2018 21:03:36 +0800

I have constructed my blog by Jekyll on Github Pages, I thought I should write an article to introduce how to make your own blog website by Jekyll on Github Pages.

I would separate the contents into short blogs, reading long article is always easy to make people feel weary. 😴

This article will introduce how to create your blog based on Github Pages. Next article I will introduce Jekyll and how to use it to enrich your blog.

Why is Github Pages?

As an open-source code platform and worldwide developer community, Github has more than millions of users now. As one of them, I have much time spent on Github and Github also provides the personal homepage service (Github Pages). It is possible to build your blog on it. To me, it is a good idea to move my blog to Github Pages with the benefit of saving lots of time on the maintenance of the blogs on different websites. That’s why I’d like to do this thing.

How to do?

The way of hosting your website on Github Pages is static access, but it can fully meet my needs. So here comes the question, how do we build blog on Github Pages? Fortunately, the procedure is simple, just need to create a new repository and upload your website code onto it. After then you can access it at any time and anywhere.

To learn how to build your website on Github, a quick way is to refer to the official help document.

Github Pages

Here is a short summary:

apply for a Github account, you can skip this step if you already have.
create a new repository as below.
fill the name of repository，please note the name must be your website’s URL and ending with Github.io!
click create repository button，the repository will be created.
open the settings page of your new repository, find your website URL, and record it. Input your website URL into the web browser and then you will see your website’s homepage. It is an empty page at the beginning. To add content, you need to create an index.html file in your repository directory. Write what you want into it and submit the modification, you will see the contents you added.

For now, all work is done! you’ve got your website with some simple elements. If you want to enrich your website, You need to do some extra works. A fast way is to use the Github Pages theme. Github Pages provides many beautiful theme templates. You can use them directly to create your own. The theme templates on Github are mostly built based on Jekyll, so it is necessary to understand Jekyll. However, the introduction of Jekyll is not discussed in this paper. I’ll write another article for the introduction of Jekyll.

See you soon. 🙂

Powershell and CMD

David Dong — Sat, 09 Dec 2017 20:44:05 +0800

Although both of them are command-line consoles of the Windows system, and they have a similar outlook and Powershell can be treated as an updated version of CMD, the intrinsic design model of them is different.

Powershell is based on Microsoft .NET architect and re-designs the shell command. The biggest difference is that PowerShell is built on the basis of the .Net framework common language runtime (CLR) and .Net framework, which accepts and returns .Net framework objects. In other words, PowerShell’s pipe passes a .Net object instead of a raw string. That brings compatibility and convenience for Powershell.

Below I list some points of the difference.

object-oriented: Powershell support OOP.
compatibility: based on .NET framework, be compatible with other Windows platform applications, such as executable (.exe), batch bat, and VB script.
good programming experience: can use .NET library, syntax is very similar to advanced programming languages.
scalability: Powershell has become a platform. In addition to Microsoft, Amazon’s cloud platform management and Dell’s out-of-hand management also provide PowerShell-based management components. PowerShell has become a standard, a specification.

All of the above advantages that CMD doesn’t have. If you are familiar with .NET, you can use Powershell to do anything that .NET does.

Android Aidl 开发

David Dong — Wed, 01 Nov 2017 17:13:44 +0800

犹豫了几天，觉得还是把这部分内容写一下吧。虽然在网上已经一大堆了，但是写作本身也是一个查漏补缺的过程。通过对知识的总结归纳可以进一步加深理解。

什么是AIDL?

如果你仔细研读过Android的源码，你会发现其中有大量的.aidl类型的文件。这些文件在android 中起到什么样的作用？

要理解.aidl, 需要从跨进程通信（IPC）说起，我们知道，在Android系统中，每个进程都有自己独立的地址空间，所有的资源分配都是在独立的内存空间内完成的。因此每个进程都和其他进程分隔了开来，每个APP都是一个独立的进程。有时候我们有应用间进行互动的需求，比如两个APP之间传递数据或者APP与系统服务进程之间的交互，任务委托等。

Android中有很多种跨进程通信的方式，比如使用 Bundle, ContentProvider，Socket和BroadcastReceiver等。 AIDL也是其中的一种实现方式。通过AIDL，可以在一个进程中获取另一个进程的数据和调用其释放出来的方法，从而满足进程间通信的需求。

AIDL 全称是 Android Interface Description Lanuage, 即Android接口描述语言。从名字上可以看出 AIDL 主要是定义了接口。AIDL接口定义语言的语法十分简单，和Java很相似，仅存在以下几点差异：

AIDL文件以 .aidl 为后缀名
AIDL支持的数据类型分为如下几种：
- 基本数据类型：byte、char、short、int、long、float、double、boolean String，CharSequence
- 自定义的数据类型必须实现Parcelable接口
- List 类型。List承载的数据必须是AIDL支持的类型，或者是其它声明的AIDL对象
- Map类型。Map承载的数据必须是AIDL支持的类型，或者是其它声明的AIDL对象
除了基本类型数据外，其他的数据类型全部需要显式导包，即使在同一个包内也需要导包。

熟悉Java语法的开发者在使用AIDL接口时应该没有任何障碍。

如何使用AIDL

AIDL 使用起来并不复杂，编程人员只需要在.aidl文件定义好接口，Android plafrom-tools会自动生成基于接口的实现类，该类为一个静态抽象类（后面会详细介绍），服务端代码只需要实现接口的业务即可。因此我们也可以将.aidl 理解为一个模板，通过这个模板生成一些跨进程通信必要的代码，开发者只需要关注实现业务内容即可，而无需花精力在实现IPC通信上面。

下面来具体说明如何实现AIDL编程。

首先AIDL文件分为两类，一类用来声明实现了Parcelable接口的数据类型，以供其他AIDL文件使用那些非默认支持的数据类型。还有一类是用来定义接口方法，声明要暴露哪些接口给客户端调用。这两种的AIDL文件编写方法是不同的。
方向TAG，对于非基本数据类型，在声明AIDL接口的时候需要使用方向TAG来声明数据的流动方向，其中
- in 表示数据只能由客户端流向服务端
- out 表示数据由服务端流向客户端
- inout 表示数据可以在服务端和客户端之间双向流动

服务端编程

服务端编程步骤如下：

生成一个.aidl文件，文件里声明自定义的对象类型，该数据类一定要实现 parcelable接口（非8种一般数据类型）。编写的 AIDL文件示例如下

关于该文件创建，需要注意以下几点：
a. 右键点击包名，New->AIDL文件
b .aidl文件的名字要和后续定义的数据类名一致才可以。不然会报 cannot find the import xxx.class。
c. 在Android Studio中要先创建AIDL文件然后创建数据类文件，否则会提示命名重复错误。
d. parcelable 注意首字母P要小写，这里和要实现的Parcelable接口不同，不然会报 interface declaration, parcelable declaration, AidlTokenType.import or AidlTokenType.package 5

编写自定义的数据类，该类一定要实现 Parcelable 接口（注意大写P）。
为自定义的接口编写 AIDL 文件，文件名随便定。
如果使用了非一般数据类型，接口要加 In/out/inout
一定要引用使用的数据类的包路径进来（导包），即使在一个大包下也要导包。
make project 后生成 AIDL的Java文件，路径在 build/generated/下，观察生成的Java文件发现生成了两个类，其中静态抽象类.Stub继承了Binder类并实现了AIDL接口的方法，注意asBinder()方法。
AIDL/自定义数据类完成后，编写 service 类，类内部定义一个 IBinder/.Stub类型的成员mBinder，实现其内部的方法，并且在service的onBind方法内返回该mBinder。

客户端程序

拷贝服务端的 .aidl目录到client项目相同目录下，包名保持不变。
拷贝服务端的数据类型类到 client相同目录下，包名保持不变。
编写业务代码，代码中需要生成一个 AIDL接口的对象和一个ServiceConnection 对象。
将本地的AIDL接口对象通过onServiceConnected方法绑定到远端的Binder对象。
通过bindService方法启动远端服务，远端服务启动后，本地的AIDL对象会绑定到远端的Binder对象。
绑定后就可以通过本地Binder对象来调用远端的接口方法了。

以上步骤可以看出AIDL接口实际上是远端服务提供的一个钩子，本地应用端要访问远端的服务，首先要先绑定远端服务的钩子（也就是Stub静态类提供的asInterface方法IBookInterface.Stub.asInterface(iBinder)），绑定后即可调用远端的服务。

Use bcdedit.exe in Windows

David Dong — Sat, 08 Jul 2017 21:11:09 +0800

BCDEDIT - Boot Configuration Data Store Editor.
Bcdedit.exe is a command-line tool for managing boot configuration data (BCD). To use this tool needs the administrator permission to start a command-line window.

The Bcdedit.exe command-line tool modifies the boot configuration data store. The boot configuration data store contains boot configuration parameters and controls how the operating system is booted. These parameters were previously in the Boot.ini file (in BIOS-based operating systems) or in the nonvolatile RAM entries (in Extensible Firmware Interface-based operating systems). You can use Bcdedit.exe to add, delete, edit, and append entries in the boot configuration data store.

In the command line, enter the command bcdedit /? to view all bcdedit related operations.

The command parameters are as below.

Commands that operate on a store
================================
/store          Used to specify a BCD store other than the current system default.
/createstore    Creates a new and empty boot configuration data store.
/export         Exports the contents of the system store to a file. This file
                can be used later to restore the state of the system store.
/import         Restores the state of the system store using a backup file
                created with the /export command.
/sysstore       Sets the system store device (only affects EFI systems, does
                not persist across reboots, and is only used in cases where
                the system store device is ambiguous).

Commands that operate on entries in a store
===========================================
/copy           Makes copies of entries in the store.
/create         Creates new entries in the store.
/delete         Deletes entries from the store.
/mirror         Creates mirror of entries in the store.

Run bcdedit /? ID for information about identifiers used by these commands.

Commands that operate on entry options
======================================
/deletevalue    Deletes entry options from the store.
/set            Sets entry option values in the store.

Run bcdedit /? TYPES for a list of datatypes used by these commands.
Run bcdedit /? FORMATS for a list of valid data formats.

Commands that control output
============================
/enum           Lists entries in the store.
/v              Command-line option that displays entry identifiers in full,
                rather than using names for well-known identifiers.
                Use /v by itself as a command to display entry identifiers
                in full for the ACTIVE type.

Running "bcdedit" by itself is equivalent to running "bcdedit /enum ACTIVE".

Commands that control the boot manager
======================================
/bootsequence   Sets the one-time boot sequence for the boot manager.
/default        Sets the default entry that the boot manager will use.
/displayorder   Sets the order in which the boot manager displays the
                multiboot menu.
/timeout        Sets the boot manager time-out value.
/toolsdisplayorder  Sets the order in which the boot manager displays
                    the tools menu.

Commands that control Emergency Management Services for a boot application
==========================================================================
/bootems        Enables or disables Emergency Management Services
                for a boot application.
/ems            Enables or disables Emergency Management Services for an
                operating system entry.
/emssettings    Sets the global Emergency Management Services parameters.

Command that control debugging
==============================
/bootdebug      Enables or disables boot debugging for a boot application.
/dbgsettings    Sets the global debugger parameters.
/debug          Enables or disables kernel debugging for an operating system
                entry.
/hypervisorsettings  Sets the hypervisor parameters.

Command that control remote event logging
=========================================
/eventsettings  Sets the global remote event logging parameters.
/event          Enables or disables remote event logging for an     
                operating system entry.

Normally I use below commands in the development.

turn off digital signature.

bcdedit /set nointegritychecks on

turn on digital signature

bcdedit /set nointegritychecks off

enable kernel debug

bcdedit /set {default} DEBUG YES

enable/disable test signature

bcdedit /set TESTSIGNING ON | OFF

set a serial debug

bcdedit /dbgsettings serial baudrate:115200 debugport:1

set network debug

bcdedit /dbgsettings net hostip:192.168.xx.xx port:50000 key:1.2.3.4

hostip: host IP address, port: host port,key: password.

copy a entry

bcdedit /copy {current} /d “your name”

几种常见的排序算法

David Dong — Mon, 19 Sep 2016 19:04:30 +0800

记得很多年以前，在我研究生毕业找工作那会儿，面试的时候被面试官问到过两次关于排序算法的问题。这件事至今给我的印象比较深刻，因为很少有在不同的面试过程中会被问到相同的问题。由此可见排序算法在面试中占有很大的比重，其实这也是考察程序员的一个基础。前几天在招聘的过程中又想起了这件事，觉得虽然这个问题比较基础，但是还是有必要整理总结一下的。

大多数的人应该都能回答出冒泡排序和选择排序的算法，我想这可能是因为国内目前的教材中（比如「C语言编程基础」）这两种为例子的比较多的缘故吧。其实在后来的工作中了解到不止这两种算法，还有一些其他的算法也可以实现。当然能够回答出冒泡排序和选择排序也应该是合格的。下面我就来总结一下目前我所知道的常见的几种排序算法。

1. 冒泡排序

我们就从最常见的冒泡排序来说起。为了统一起见，本文的排序都是以从小到大的序列为准。文中的例子都以C语言代码编写，对python和Java的源码感兴趣的同学，可以在这里找到源码。

冒泡排序规则是：从左到右，数组中相邻的两个元素进行比较，将较大的放到后面。
按照这个原则，我们编写代码。

C语言实现

void sort_bubble(int *a, int num)
{

    int i = 0;
    int j = 0;
    int temp = 0;

    for (i = 0; i < num - 1; i++)
    {
        for (int j = 0; j < num - i - 1; j++)
        {
            if (a[j] > a[j + 1])
            {
                temp = a[j];
                a[j] = a[j + 1];
                a[j + 1] = temp;
            }
        }
        printData(a, num);
    }
}

打印结果如下

initial data:
17,6,19,45,1,2,23,5,4,11,3,
options:1
starting bubble sort...
6,17,19,1,2,23,5,4,11,3,45,
6,17,1,2,19,5,4,11,3,23,45,
6,1,2,17,5,4,11,3,19,23,45,
1,2,6,5,4,11,3,17,19,23,45,
1,2,5,4,6,3,11,17,19,23,45,
1,2,4,5,3,6,11,17,19,23,45,
1,2,4,3,5,6,11,17,19,23,45,
1,2,3,4,5,6,11,17,19,23,45,
1,2,3,4,5,6,11,17,19,23,45,
1,2,3,4,5,6,11,17,19,23,45,

结果分析
数组长度11，一共比较了10轮。第一轮比较结束后序列中最大数45从序列的最右边冒出来。第二轮比较后，所有数中第二大的那个数23就会浮到倒数第二个位置……就这样一轮一轮地比较，最后实现从小到大排序。

轮次	比较次数	结果
第1轮	10	6,17,19,1,2,23,5,4,11,3,`45`
第2轮	9	6,17,1,2,19,5,4,11,3,`23`,45
第3轮	8	6,1,2,17,5,4,11,3,`19`,~~23,45~~
第4轮	7	1,2,6,5,4,11,3,`17`,~~19,23,45~~
第5轮	6	1,2,5,4,6,3,`11`,~~17,19,23,45~~
第6轮	5	1,2,4,5,3,`6`,~~11,17,19,23,45~~
第7轮	4	1,2,4,3,`5`,~~6,11,17,19,23,45~~
第8轮	3	1,2,3,`4`,~~5,6,11,17,19,23,45~~
第9轮	2	1,2,`3`,~~4,5,6,11,17,19,23,45~~
第10轮	1	1,`2`,~~3,4,5,6,11,17,19,23,45~~

时间/空间复杂度
由于我们要重复执行n次冒泡，每次冒泡要执行n次比较，也就是O(n^2)。空间复杂度是O(n)。

2. 选择排序

选择排序规则是：从第一个位置开始比较，找出最小的，和第一个位置互换，开始下一轮。第二轮从第二个位置开始查找，然后找出剩余序列中最小的，和第二个位置互换，然后重复之前的操作，依次比较，直至所有数都排列完毕。

C语言实现

void sort_selection(int *a, int num)
{

    int i = 0;
    int j = 0;
    int temp = 0;
    int min;

    for (i = 0; i < num - 1; i++)
    {
        min = i;
        for (j = i + 1; j < num; j++)
        {
            if (a[j] < a[min])
            {
                min = j;
            }
        }
        temp = a[i];
        a[i] = a[min];
        a[min] = temp;
        printData(a, num);
    }
}

打印结果

initial data:
17,6,19,45,1,2,23,5,4,11,3,
options:2
starting selection...
1,6,19,45,17,2,23,5,4,11,3,
1,2,19,45,17,6,23,5,4,11,3,
1,2,3,45,17,6,23,5,4,11,19,
1,2,3,4,17,6,23,5,45,11,19,
1,2,3,4,5,6,23,17,45,11,19,
1,2,3,4,5,6,23,17,45,11,19,
1,2,3,4,5,6,11,17,45,23,19,
1,2,3,4,5,6,11,17,45,23,19,
1,2,3,4,5,6,11,17,19,23,45,
1,2,3,4,5,6,11,17,19,23,45,

结果分析
也是一共比较了10轮。第一轮比较结束后序列中最小数a[4] 和a[0]互换。第二轮从a[1]开始比较，因为a[0]已经是最小的数了，比较完成后，a[1]~a[10]中最小的数a[5]和a[1]互换。依次论推，直到把数据排列好。

轮次	比较次数	结果
第1轮	10	`1`,6,19,45,`17`,2,23,5,4,11,3
第2轮	9	1,`2`,19,45,17,`6`,23,5,4,11,3
第3轮	8	~~1,2,~~`3`,45,17,6,23,5,4,11,`19`
第4轮	7	~~1,2,3,~~`4`,17,6,23,5,`45`,11,19
第5轮	6	~~1,2,3,4,~~`5`,6,23,`17`,45,11,19
第6轮	5	~~1,2,3,4,5,~~`6`,`23`,17,45,11,19
第7轮	4	~~1,2,3,4,5,6,~~`11`,17,45,`23`,19
第8轮	3	~~1,2,3,4,5,6,11,~~17,45,23,19
第9轮	2	~~1,2,3,4,5,6,11,17,~~`19`,23,`45`
第10轮	1	~~1,2,3,4,5,6,11,17,19,~~23,45

时间/空间复杂度
每次要找一遍最小值，最坏情况下找n次，这样的过程要执行n次，所以时间复杂度还是O(n^2)。空间复杂度是O(n)。

3. 快速排序

快速排序规则是：选择序列中的一个数为基准数，序列中的数逐个与之比较，小的放到它的左边，大的放大它的右边，然后再对左右区间递归执行上述步骤，直至整个序列有序。

C语言实现

void sort_quick(int *a, int start, int end)
{
    int i = start;
    int j = end;
    int temp = 0;
    int base = a[i];

    if (start < end)
    {
        while (i != j)
        {
            while (i < j && a[j] >= base)
                j--;

            while (i < j && a[i] <= base)
                i++;

            temp = a[i];
            a[i] = a[j];
            a[j] = temp;

            printData(a, end + 1);
        }

        a[start] = a[i];
        a[i] = base;

        sort_quick(a, start, i - 1);
        sort_quick(a, i + 1, end);
    }
}

打印结果

initial data:
17,6,19,45,1,2,23,5,4,11,3,
options:3
17,6,3,45,1,2,23,5,4,11,19,
17,6,3,11,1,2,23,5,4,45,19,
17,6,3,11,1,2,4,5,23,45,19,
17,6,3,11,1,2,4,5,23,45,19,
search done:5,6,3,11,1,2,4,17,23,45,19,
5,4,3,11,1,2,6,
5,4,3,2,1,11,6,
5,4,3,2,1,11,6,
search done:1,4,3,2,5,11,6,
1,4,3,2,
search done:1,4,3,2,
1,4,3,2,
search done:1,2,3,4,
1,2,3,
search done:1,2,3,
1,2,3,4,5,11,6,
search done:1,2,3,4,5,6,11,
1,2,3,4,5,6,11,17,23,19,45,
1,2,3,4,5,6,11,17,23,19,45,
search done:1,2,3,4,5,6,11,17,19,23,45,
after sort, data:
1,2,3,4,5,6,11,17,19,23,45,

结果分析
以17(a[0])做为基准数，先从序列尾部（右边）开始比较，大家可以思考一下为什么要先从右边开始，如果从左边开始会出现什么问题？ 右边第一个数字3(a[10])小于基准数，因此需要调换位置（需要调换到的目的位置要从头部开始查找），尾部的比较停止在这个位置，然后从头部开始比较（左边），寻找大于基准数的元素的位置，找到该位置后与刚才的元素3对调。我们看到找到的是元素19（a[2]）,将a[2]与a[10]对调。执行完这一步后序列元素如打印结果中第一行所示。

序列	元素
初始序列	17,6,19,45,1,2,23,5,4,11,3
`第一次交换后`	17,6,`3`,45,1,2,23,5,4,11,`19`

然后从尾部接着查找下一个小于基准数的元素，重复刚才的步骤,直到头尾部相遇为止。经过这一轮排序后，整个序列已经按照基准数分为了两组（如下），左边的小于基准数17,右边的大于基准数。最后将最后相遇的位置与a[0]对调。

序列	元素
初始序列	17,6,19,45,1,2,23,5,4,11,3
`第一轮交换后`	`5,6,3,11,1,2,4,`17,`23,45,19`

然后对左右两组分别递归调用排序，直到所有的数排列有序为止。

时间/空间复杂度

4. 插入排序

插入排序规则是：对于每个未排序数据，在已排序序列中从后向前扫描，找到相应位置并插入。

C语言实现

void sort_insert(int *a, int num)
{
    int i, j = 0;
    int temp;

    for (i = 1; i < num; i++)
    {
        if (a[i] < a[i - 1])
        {
            temp = a[i];
            for (j = i; j >= 0; j--)
            {
                if (j > 0 && a[j - 1] > temp)
                {
                    a[j] = a[j - 1];
                }
                else
                {
                    a[j] = temp;
                    break;
                }
            }
        }
        printf("search done:");
        printData(a,num);
    }
}

打印结果

initial data:
17,6,19,45,1,2,23,5,4,11,3,
options:4
starting insert sort...
search done:6,17,19,45,1,2,23,5,4,11,3,
search done:6,17,19,45,1,2,23,5,4,11,3,
search done:6,17,19,45,1,2,23,5,4,11,3,
search done:1,6,17,19,45,2,23,5,4,11,3,
search done:1,2,6,17,19,45,23,5,4,11,3,
search done:1,2,6,17,19,23,45,5,4,11,3,
search done:1,2,5,6,17,19,23,45,4,11,3,
search done:1,2,4,5,6,17,19,23,45,11,3,
search done:1,2,4,5,6,11,17,19,23,45,3,
search done:1,2,3,4,5,6,11,17,19,23,45,
after sort, data:
1,2,3,4,5,6,11,17,19,23,45,

结果分析
从a[0]开始，逐个与下个相邻元素比较，如果a[i]大于a[i+1]，那么就把a[i+1]放到a[i]之前的序列中进行排序。如第三轮中交换后元素1被交换到序列6,17,19,45中进行排序。依次重复，直到所有数都有序排列为止。

序列	元素
初始序列	6,17,19,45,`1`,2,23,5,4,11,3
`交换后`	`1`,6,17,19,45,2,23,5,4,11,3

时间/空间复杂度
因为要选择n次，而且插入时最坏要比较n次，所以时间复杂度同样是O(n^2)。空间复杂度是O(n)。

5. Shell排序

Shell排序规则是：分组插入排序, 将数组按照一定步长分成几个子数组,对子数组（列）分别进行插入排序，重复这过程，不过每次用更小的步长-更长的列来进行。

C语言实现

void sort_shell(int *a, int num){

    int i,j = 0;
    int temp = 0;
    int scale = round(num);
    while(scale > 0){

        for(i=scale;i<num;i++){
            temp = a[i];
            j = i;
            while ( j >= scale && a[j-scale] > temp ){
                a[j] = a[j-scale];
                j = j - scale;
            }
            a[j] = temp;
        }
        scale = round(scale/2);
        printf("search done:");
        printData(a,num);
    }
}

打印结果

initial data:
17,6,19,45,1,2,23,5,4,11,3,
options:5
starting shell sort...
search done:17,6,19,45,1,2,23,5,4,11,3,
search done:2,6,5,4,1,3,23,19,45,11,17,
search done:1,3,2,4,5,6,17,11,23,19,45,
search done:1,2,3,4,5,6,11,17,19,23,45,
after sort, data:
1,2,3,4,5,6,11,17,19,23,45,

结果分析
以步长为5开始进行排序，可以通过将这列表放在有5列的表中来更好地描述算法，这样他们就应该看起来是这样：

17|  6| 19| 45|  1
 2| 23|  5|  4| 11
 3|

对这5列进行插入排序后，序列变为

 2|  6|  5|  4|  1
 3| 23| 19| 45| 11
17|

将这5列连接在一起的原始序列为2,6,5,4,1,3,23,19,45,11,17,然后再对该序列以步长2排序。方法相同，排序完成后的数列为1,3,2,4,5,6,17,11,23,19,45。重复该步骤直到步长为1。

时间/空间复杂度
希尔排序的时间复杂度受步长的影响。

聊一聊开源协议那些事

David Dong — Sat, 25 Jun 2016 15:34:40 +0800

说到开源代码，有些人认为既然都已经开源了，那自然不需要声明任何软件的许可协议。有意思的是，事实正好相反，如果你想把代码分享出来，最好还是选择一个合适的开源协议，这样别人才可以放心的使用。让我们先看一看什么是开源协议。

什么是开源协议

我们讲的开源协议通常指的是软件的许可协议(License)，主要内容其实就是界定了软件授权人和被授权人的权利和义务。打个比方，作为软件开发者的你，想开放哪些授权给软件的使用者？后续是否会对软件提供相应的维护以及对于该软件造成的不良后果承担相应的责任？而作为软件的使用者，在使用软件的过程中哪些事情可以做，哪些事情不可以做，哪些事情必须做？软件的许可协议正是用来处理此类问题的法律文书，具有法律效力。

以下是开源协议在维基百科的定义。

开源许可是一种计算机软件和其他产品的许可类型，允许使用、修改或在定义的条款和条件下使用、修改或共享的源代码、蓝图和设计。这允许终端用户和商业公司对源代码、图纸或设计进行审查和修改，以满足自己的定制、好奇心或故障排除的需要。开源许可的软件大多是免费的，尽管这并不一定是必须的。许可证只允许非商业的重新分配或修改个人使用的源代码，通常不被认为是开源许可。然而,开源许可可能会有一些限制,尤其是对软件的起源的表达,比如要求保留作者的名字和代码中的版权声明,或要求重新分配授权软件只有在相同的许可(如copyleft许可证)。一组流行的开源软件许可证是由开源计划(OSI)根据其开源定义(OSD)批准的。

而有些人会说，如果我不申明任何许可协议会怎么样？法律上规定，不声明任何许可协议的话等同于保留所有的权利和义务，也就是版权保留(copyright)。这种情况下别人拿到你的代码后基本上什么也做不了，不能复制，修改，发布以及再授权甚至销售等。除非得到你的’特殊’的授权。这个就和软件开源的初衷南辕北辙了。所以说如果想要分享你的软件的话最好还是声明一个许可协议。

现在有很多比较流行的开源许可协议，比如 GPL、BSD、LGPL、MIT、Apache 等等。这些协议界定的权利和义务有所不同，你可以仔细研究选择一个适合你的来使用。当然如果你不嫌麻烦的话自己写一个许可协议也是可以的，同样具有法律效力。😉 很多知名的大公司都有自己的软件授权许可协议。

不同的开源许可协议

刚才讲过，目前流行的开源许可协议对于授权和责任上的界定是有所不同的。下面我们就分析一下它们之间都有哪些差异。

对比这些协议的差异其实是一件很耗费时间和精力的事情，幸运的是我从网上找来了一张图片，可以很清楚简明的把这件事讲清楚，也能节省我不少的时间，好让我有时间喝杯 ☕。
图片来源：http://www.ruanyifeng.com/blog/2011/05/how_to_choose_free_software_licenses.html

以上所有的协议都是免费的，选择时根据自身的需要选择即可。

网上看到过有人专门做过分析，目前Github上使用率最高的许可证是MIT，后面依次是Apache 2.0、GPL 2.0、GPL 3.0和bsd-3-clause。对以上的分析结果不做置评，不过就我自身的体会，在Github上见到使用最多的许可协议确实是 MIT。多数个人的作品和代码用 MIT 的比较多，我自己在Github上的几个项目也都是使用了 MIT 的许可协议。

MIT 协议

既然讲到了 MIT 协议，那我们就再来看一下这个协议的内容。
下面以我自己的一个项目的 MIT 协议为例。这份协议基于Github的 MIT 协议模板生成，内容为一份标准的官方协议，只是权利人和时间是填写的我的。
原文如下：

MIT License

Copyright (c) 2016 David Dong

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

简单来讲，就是这份协议明确的划定了我作为软件的权利所有人授予了软件的使用者哪些授权和我应该承担的责任。下面我们就来划一下重点。

这一行声明了该软件的权利人和发布时间。这也是 MIT 协议中唯一需要填写的地方。如果只包含一个年份，说明这是首次发布的年份。如果包含时间段，则第一个年份为第一次发布的年份，第二个是当前版本发布的年份。

授予的软件使用者权利

including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software.

也就是说使用MIT协议我授权给软件的使用者可以使用，复制，修改，发布，分发，再授权以及销售软件copy的权利。

免责声明

THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY

使用 MIT

好了，最后再来说一下如何使用 MIT 协议，基本上 MIT 是最宽松的协议，如果你 fork 了一份代码，作者声明了 MIT 许可，那你只需要做一件事，即保留作者的版权和许可证信息。其余的你可以随便使用，修改，分发该代码。

保留作者的版权和许可证信息意味着：

你发布的代码中要包含原作者的LICENSE文件，这个文件有完整的 MIT 许可证，其中也会有作者的版权信息。
保留源码里面的版权和许可信息头。
如果你发行的是二进制文件（可执行软件），那就要在软件的某个界面上说明或者Readme文件里声明原作者的 MIT 许可。

A way to fix .gitignore doesn't work

David Dong — Sat, 06 Jun 2015 21:41:29 +0800

You may have run into situations where .gitignore doesn’t work.😕
During the project development, you want to add some directors or files into the ignore rule and you specified them at the .gitignore. You committed and checked the status, unfortunately, you found they were still being tracked.

That is because the .gitignore can only ignore those files that were not originally tracked. If some files have been submitted before and have been included in version management, modifying .gitignore is invalid. The solution is to delete the local cache (change it to untrack) and resubmit .gitignore file.

The git command is like this.

git rm -r --cached .
git add .
git commit -m "refresh .gitnore"
git push -u origin master

In short, .gitignore just ignores files that are not staged (cached). For the files that have been staged, when adding .gitignore, they must be removed from the staged before they can be ignored.

Read/Write system property in Android APP

David Dong — Sat, 11 Apr 2015 23:48:12 +0800

Recently I need to read system property in my Android app development. I checked the Android documents and found that Android has a Systemproperties class to handle this kind of requirement. However, Systemproperties class isn’t a public class of SDK, which means it is invisible to the app developer. My understanding is that Google wants this class to be used for Android source developers rather than application developers.

If I use this class as a source developer, I have to raise my program with root permission and re-build the whole system.

So I applied a trick way to access the Systemproperties method with the help of the Java reflect mechanism.

I studied the source code of SystemProperties.java and found the methods it implemented.

SystemProperties.java

private static native String native_get(String key);
private static native String native_get(String key, String def);
private static native int native_get_int(String key, int def);
private static native long native_get_long(String key, long def);
private static native boolean native_get_boolean(String key, boolean def);
private static native void native_set(String key, String def);
private static native void native_add_change_callback();
private static native void native_report_sysprop_change();

This class defined the set/get methods for the different data types. What I need is the get() method of return a String type.

The rest work is straightforward, I wrote the below codes and inserted them into my class.

...
import java.lang.reflect.Method;
import android.support.annotation.NonNull;
import android.support.annotation.Nullable;
...

public class MainActivity extends DisabledNavigationActivity {
     private Logger mLogger = new Logger(getClass().getSimpleName());
     private static final long VIBRATE_MS_IMAGE = 50;
     private Handler mHandler = new Handler(Looper.getMainLooper());
 
     ...

	 private final String vendorKey = "my_property";

     ...

	private interface PropertiesKeys {
        String VENDOR = "persist.vendor.sys.fp.vendor";
    }

	private String getStringProperty(
            @Nullable Method systemPropertiesGetString,
            @NonNull String propertyKey) {
        final String defaultValue = "";
        if (null == systemPropertiesGetString) {
            return defaultValue;
        }
        try {
            return (String) systemPropertiesGetString.invoke(
                    null, // No object when calling a static method.
                    propertyKey,
                    defaultValue);

        } catch (Exception e) {
            mLogger.e(Logger.format(
                  "Failed to read system property (%s): %s",
                  propertyKey,
                 e.getMessage()));
        }
        return defaultValue;
    }

    ....

     @Override
     protected void onCreate(Bundle savedInstanceState) {
         super.onCreate(savedInstanceState);
         mLogger.enter("onCreate");
         setContentView(R.layout.activity_main);
 
		Method sysPropGetString = null;
		try {
            // The SystemProperties class is not public in the SDK - access via reflection.
            final Class systemProperties = Class.forName("android.os.SystemProperties");
            sysPropGetString = systemProperties.getMethod("get",String.class,String.class);
        } catch (Exception e) {
            mLogger.e(Logger.format(
                    "Failed to get method system properties: %s",
                    e.getMessage()));
        }

		String propStr = getStringProperty(sysPropGetString, PropertiesKeys.VENDOR);

		mLogger.d("the vendor is " + propStr);

		if(!propStr.equal(vendorKey)){
			...
		}
	
		...

Getevent command usage

David Dong — Wed, 25 Feb 2015 20:33:04 +0800

Android provides an ADB command getevent to obtain input events, such as obtaining key reporting events, obtaining touch screen reporting events, etc.

Users can use this command under the ADB console.

Usage

The usage of this command is:

adb shell getevent [options] [device]

We input adb shell getevent -h on the CMD command line and will get the help information about this command.

Usage: getevent [-t] [-n] [-s switchmask] [-S] [-v [mask]] [-d] [-p] [-i] [-l] [-q]
[-c count] [-r] [device]
  -t: show time stamps
  -n: don't print newlines
  -s: print switch states for given bits
  -S: print all switch states
  -v: verbosity mask (errs=1, dev=2, name=4, info=8, vers=16, pos. events=32, 
      props=64)
  -d: show HID descriptor, if available
  -p: show possible events (errs, dev, name, pos. events)
  -i: show all device info and possible events
  -l: label event types and names in plain text
  -q: quiet (clear verbosity mask)
  -c: print given number of events then exit
  -r: print rate events are received

To display which input devices are on the system, use

adb shell getevent

the command will return all of the input devices that the system used. For below my example, there are 7 input devices that registered at dev/input.

add device 1: /dev/input/event6
  name:     "sdm670-mtp-snd-card Button Jack"
add device 2: /dev/input/event5
  name:     "sdm670-mtp-snd-card Headset Jack"
add device 3: /dev/input/event4
  name:     "capsense_bottom"
add device 4: /dev/input/event3
  name:     "capsense_top"
add device 5: /dev/input/event2
  name:     "himax-touchscreen"
add device 6: /dev/input/event0
  name:     "qpnp_pon"
add device 7: /dev/input/event1
  name:     "gpio-keys"

Below are some examples, we can find the touchscreen input event is registered at /dev/input/event2, so we use the touchscreen as an input device for the interpretation.

non-parameter:

C:\WINDOWS\System32>adb shell getevent /dev/input/event2
0003 0030 0000000f
0003 0032 0000000f
0003 003a 0000000f
0003 0035 00000154

Here the first column represents the EVENT_TYPE, the second column displays the EVENT_CODE and the third column is EVENT_VALUE.

-t: display timestamp

C:\WINDOWS\System32>adb shell getevent -t /dev/input/event2
[    1564.568377] 0003 0030 0000000f
[    1564.568377] 0003 0032 0000000f
[    1564.568377] 0003 003a 0000000f

-r: print report rate

C:\WINDOWS\System32>adb shell getevent -r /dev/input/event2
0003 0032 00000027
0003 003a 00000027
0000 0000 00000000 rate 135

-p: print the event informaiton supported by the device.

C:\WINDOWS\System32>adb shell getevent -p /dev/input/event2
add device 1: /dev/input/event2
  name:     "himax-touchscreen"
  events:
    KEY (0001): 0066  008b  009e  00d9  014a  0244
    ABS (0003): 002f  : value 0, min 0, max 9, fuzz 0, flat 0, resolution 0
                0030  : value 0, min 0, max 200, fuzz 0, flat 0, resolution 0
                0032  : value 0, min 0, max 200, fuzz 0, flat 0, resolution 0
                0035  : value 0, min 0, max 719, fuzz 0, flat 0, resolution 0
                0036  : value 0, min 0, max 1599, fuzz 0, flat 0, resolution 0
                0039  : value 0, min 0, max 65535, fuzz 0, flat 0, resolution 0
                003a  : value 0, min 0, max 200, fuzz 0, flat 0, resolution 0
  input props:
    INPUT_PROP_DIRECT

-i: similar to -p, but will contain more information.

C:\WINDOWS\System32>adb shell getevent -i /dev/input/event2
add device 1: /dev/input/event2
  bus:      0000
  vendor    0000
  product   0000
  version   0000
  name:     "himax-touchscreen"
  location: ""
  id:       ""
  version:  1.0.1
  events:
    KEY (0001): 0066  008b  009e  00d9  014a  0244
    ABS (0003): 002f  : value 0, min 0, max 9, fuzz 0, flat 0, resolution 0
                0030  : value 0, min 0, max 200, fuzz 0, flat 0, resolution 0
                0032  : value 0, min 0, max 200, fuzz 0, flat 0, resolution 0
                0035  : value 0, min 0, max 719, fuzz 0, flat 0, resolution 0
                0036  : value 0, min 0, max 1599, fuzz 0, flat 0, resolution 0
                0039  : value 0, min 0, max 65535, fuzz 0, flat 0, resolution 0
                003a  : value 0, min 0, max 200, fuzz 0, flat 0, resolution 0
  input props:
    INPUT_PROP_DIRECT

-l: print the event type/event code by plain text.

C:\WINDOWS\System32>adb shell getevent -l /dev/input/event2
EV_ABS       ABS_MT_TOUCH_MAJOR   0000000f
EV_ABS       ABS_MT_WIDTH_MAJOR   0000000f
EV_ABS       ABS_MT_PRESSURE      0000000f

Compare with the default output, the -l output use the plain text replace the raw code of EVENT_TYPE/EVENT_CODE/EVENT_VALUE which is more intuitive.

-c: print given number of events then exit

C:\WINDOWS\System32>adb shell getevent -c 10  /dev/input/event2
0003 0030 0000000f
0003 0032 0000000f
0003 003a 0000000f
0003 0035 0000016d
0003 0036 00000459
0003 0039 00000025
0001 014a 00000001
0000 0000 00000000
0003 0030 00000031
0003 0032 00000031

normal when we debug the touchscreen, we usually use below command to obtain the input event. This command will give all information with plain text output, which is useful for obtain the image of which input event is occurring.

C:\WINDOWS\System32>adb shell getevent -ltr  /dev/input/event2
[    2311.461360] EV_ABS       ABS_MT_TOUCH_MAJOR   0000000f
[    2311.461360] EV_ABS       ABS_MT_WIDTH_MAJOR   0000000f
[    2311.461360] EV_ABS       ABS_MT_PRESSURE      0000000f
[    2311.461360] EV_ABS       ABS_MT_POSITION_X    0000013f
[    2311.461360] EV_ABS       ABS_MT_POSITION_Y    000003e6
[    2311.461360] EV_ABS       ABS_MT_TRACKING_ID   00000026
[    2311.461360] EV_KEY       BTN_TOUCH            DOWN
[    2311.461360] EV_SYN       SYN_REPORT           00000000

The source code is at getevent.c.

Use of .PHONY in makefile

David Dong — Tue, 11 Nov 2014 23:55:17 +0800

You may have run into a situation where the name of the makefile command conflicts with the file name under the same directory.

For example, I have below code.

B_TARGET:=testapp
clean:
  rm -f $(B_TARGET)

I execute make clean, everything is ok. The target folder is removed as expected. However, if there is a file named clean in the directory. I will get a result when I execute the make clean command.

make: 'clean' is up to date.

To avoid this issue happen, use .PHONY command. The main function of .PHONY configuration item is to avoid the conflict between the specified command and the file with the same name under the project and to optimize the performance. .PHONY [command name] explicitly specifies a command name instead of the actual file name, so that the make [command name] command can be parsed and executed correctly. You don’t have to worry about if the same name file existing or not.

I change the code as this.

.PHONY clean
B_TARGET:=testapp
clean:
  rm -f $(B_TARGET)

I execute make clean and the command works this time.

How does Android dispatch Touchevent?

David Dong — Thu, 02 Oct 2014 22:17:21 +0800

Recently, I’ve been busy with my work and haven’t been able to update the blog for a long time. This time, I am going to start a discussion and summarize how Android dispatch Touchevent after received them from the touch screen device.

You may know there are some ways in the android system that can capture Touch Event.

For example, register a component’s setOnTouchListener() and implements the onTouchListener interface by override the onTouch() method. Else you can change it to setOnClickListener() and implements the onClickListener interface. Also, you can even override the view’s onTouchEvent() method directly to capture the Touch Event. But have you ever thought about what the difference is among these methods? How does android handle these methods sequentially?

Let’s start with some pieces of code.

public class MainActivity extends AppCompatActivity {

    Button btn;
    String str ="";
    int cnt =0;
    
    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);

        btn = this.findViewById(R.id.button01);

        btn.setOnTouchListener(new View.OnTouchListener() {
            @Override
            public boolean onTouch(View view, MotionEvent motionEvent) {
                cnt++;
                str+= "_onTouch()";
                switch(motionEvent.getAction())
                {
                    case MotionEvent.ACTION_DOWN:
                        str += "_down";
                        break;
                    case MotionEvent.ACTION_UP:
                        str += "_up";
                        break;
                    default:
                        str += "_others";
                        break;
                }
                Log.v("check",str + "_count:" + cnt);
                return false;
            }
        });

        btn.setOnClickListener(new View.OnClickListener() {
            @Override
            public void onClick(View view) {
                cnt++;
                str+= "_onClick()";
                Log.v("check",str + "_count:" + cnt);
            }
        });
    }
}

These pieces of code put a button on the activity and register two listeners to capture the Touchevent of the button, that is the click.
Let’s see what the result is when the button is clicked.

_onTouch()_down_count:1
_onTouch()_down_onTouch()_up_count:2
_onTouch()_down_onTouch()_up_onClick()_count:3

We can see when the button is clicked, the OnTouchListener() is triggered twice (for dwon and up) firstly and then the onClickListener() is called.
If we change the return value to “true” and re-run this pieces of code.

btn.setOnTouchListener(new View.OnTouchListener() {
            @Override
            public boolean onTouch(View view, MotionEvent motionEvent) {
                cnt++;
                str+= "_onTouch()";
                switch(motionEvent.getAction())
                {
                    case MotionEvent.ACTION_DOWN:
                        str += "_down";
                        break;
                    case MotionEvent.ACTION_UP:
                        str += "_up";
                        break;
                    default:
                        str += "_others";
                        break;
                }
                Log.v("check",str + "_count:" + cnt);
                return true;
            }
        });

We see the onClickListener() will not be called anymore.

_onTouch()_down_count:1
_onTouch()_down_onTouch()_up_count:2

The above code proves that the OnTouchListener() has high priority than onClickListener() and can block the latter to be executed.

Next, let track into the Android source code to find the root cause.

As the above pictures showed, in android internal, there are two routes

one is from outside to inside, that is from activity - > ViewGroup - > view, call the dispatchTouchEvent() method from the outside to the inside, and Android will pass the motion event parameter to the method in turn. The function of dispatchTouchEvent() is to deliver touch events. The dispatchTouchEvent() is the entrance to deliver touch events every time.
another is from inside to outside, that is to call the onTouchEvent() method from view - > ViewGroup - > activity, and Android will pass the motion event parameter to the method in turn. The function of onTouchEvent() is to handle touch events.

Nex, Let’s go into every component to check the details further.

Activity

dispatchTouchEvent() All touch events generated by touch operations on the UI will first trigger the execution of the dispatchTouchEvent() method in the activity.
The source code is as follows:

public boolean dispatchTouchEvent()(MotionEvent ev) {
    if (ev.getAction() == MotionEvent.ACTION_DOWN) {
        onUserInteraction();
    }
    if (getWindow().superdispatchTouchEvent()(ev)) {
        return true;
    }
    return onTouchEvent()(ev);
}

Activity first obtains the current window object through the getWindow () method, then calls the superdispatchTouchEvent() method of window. In fact, getWindow() returns an instance of PhoneWindow type, which will invoke PhoneWindow’s superdispatchTouchEvent() method.

@Override
public boolean superdispatchTouchEvent()(MotionEvent event) {
    return mDecor.superdispatchTouchEvent()(event);
}

mDecor is a variable of type decorview in phonewindow. Decorview represents the top-level view of the current window, which can be regarded as the root view. As can be seen from the above code, the superdispatchTouchEvent() method of decorview will be executed later.
Its source code is as follows:

public boolean superdispatchTouchEvent()(MotionEvent event) {
    return super.dispatchTouchEvent()(event);
}

In fact, dectorview inherits from FrameLayout, so it indirectly inherits from ViewGroup, so it will execute the dispatchTouchEvent() method corresponding to its parent class ViewGroup.

The above describes the transfer process of touch event from activity to ViewGroup with the help of superdispatchTouchEvent() and dispatchTouchEvent() methods. Both methods return a boolean type parameter. If true is returned, it means that the touch event is processed; otherwise, it means that the touch event is not processed. If we look at the source code of the dispatchTouchEvent() in the above activity, we will find that if the superDispatchTouchEvent() of PhoneWindow returns true, then the dispatchTouchEvent() method of activity directly returns true, indicating that the touch event has been processed by the window, so the later onTouchEvent() method of activity will not be executed. Only when the window does not handle the touch event, the activity will call the onTouchEvent() method to handle the event.

onTouchEvent() The source code of onTouchEvent() is as follows

public boolean onTouchEvent()(MotionEvent event) {
    if (mWindow.shouldCloseOnTouch(this, event)) {
        finish();
        return true;
    }
    return false;
}

Only when the touch event has not been processed by any view or ViewGroup can the activity execute its own onTouchEvent() to process the touch event.

ViewGroup

source code
dispatchTouchEvent()
When the activity receives the touch event, it will call the dispatchTouchEvent() method of ViewGroup through dectorview.
The dispatchTouchEvent() method is the entry point for ViewGroup to process touch events.

A member variable of TouchTarget type mFirstTouchTarget is defined in the ViewGroup, which is used to save the child views that handle touch events in the current ViewGroup.
First of all, the dispatchTouchEvent() method will call its own onIntercepTouchEvent() method. onIntercepTouchEvent() is used to intercept the touch event passed by the ViewGroup to its child view. If the method returns true, it means that the ViewGroup should intercept the touch event; if it returns false, it means that the ViewGroup should not intercept the touch event, and it should pass the touch event to the child view. A boolean type handled variable is defined in the dispatchTouchEvent() method to save the return value of the dispatchTouchEvent() method. If true, the touch event is processed by the current ViewGroup. Otherwise, it is not processed.

Then, only when onIntercepTouchEvent() returns false, the ViewGroup will traverse its child views in turn. It will call isTransformedTouchPointInView() method to determine whether the coordinates of the touch event carried by motion event fall within the scope of the child view. If the coordinates of the touch event fall within the scope of the child view, it means that we touch the child view within the current ViewGroup In this way, the ViewGroup will pass the coordinates of the touch event and the subview to the dispatchTransformedTouchEvent() method. In this method, the dispatchTouchEvent() method of the subview will be called. Its return value indicates whether the touch event has been handled by the self-view. If the dispatchTransformedTouchEvent() returns true, it indicates that the subview has handled the touch event. In this way, the ViewGroup will call addTouchTarget() method binds mFirstTouchTarget to the child view, and the variable alreadyDispatchedToNewTouchTarget is also set to true, indicating that a child view has handled the touch event. Once a child view handles the touch event, the ViewGroup will break out of the for loop and no longer traverse other child views. After the for loop of child view, if no child view handles touch events, mFirstTouchTarget is still null, and ViewGroup will pass null as the child parameter.

In the dispatchTransformedTouchEvent() method, the method will call the super.dispatchTouchEvent() method. Since ViewGroup inherits from view, this is equivalent to executing the dispatchTouchEvent() method in the view class, so it is likely to execute the onTouchEvent() method inherited from view by ViewGroup. The return value of dispatchTransformedTouchEvent() is used as the value of the local variable handled. The dispatchTouchEvent() method in the view class is described in detail below. After the for loop of a subview, if a subview is found to handle touch events, then the already dispatchedToNewTouchTarget() is true, which will set the local variable handled to true, that is, as long as a subview handles touch events, it indicates that the current ViewGroup has also handled touch events, and in this case, the ViewGroup will not call v The dispatchTouchEvent() method inherited from the view does not trigger the execution of the onTouchEvent() method of the ViewGroup. onIntercepTouchEvent()
As mentioned earlier, onIntercepTouchEvent() is used to intercept the transfer of touch events from ViewGroup to child views. The default implementation in ViewGroup always returns false, which means no interception. We can override this method to implement our own touch event interception logic.

onTouchEvent()
The onTouchEvent() of the ViewGroup is inherited from the onTouchEvent() method of the view. The ViewGroup has not been overridden. We will introduce the implementation logic of the onTouchEvent() method of the view in the following section.

View

source code

dispatchTouchEvent()

public boolean dispatchTouchEvent()(MotionEvent event) {
    ......

    boolean result = false;

    ......

    if (onFilterTouchEventForSecurity(event)) {
        //noinspection SimplifiableIfStatement
        ListenerInfo li = mListenerInfo;
        //如果设置了onTouchListener，那么会在此处执行onTouchListener的onTouch方法
        if (li != null && li.monTouchListener != null
                && (mViewFlags & ENABLED_MASK) == ENABLED
                && li.monTouchListener.onTouch(this, event)) {
            //如果onTouchListener的onTouch方法返回true，
            //就表示触摸事件被处理了，result就会设置为true
            result = true;
        }

        //如果触摸事件没有被onTouchListener处理，那么就会执行View的onTouchEvent()方法
        if (!result && onTouchEvent()(event)) {
            //如果onTouchEvent()返回了true，
            //就表示触摸事件被View处理了，result就被设置为了true
            result = true;
        }
    }

    ......

    return result;
}

dispatchTouchEvent() is the entry point for the view to handle touch events. In this method, the view will first check whether it has been set onTouchListener. If it has, it will call onTouchListener’s ontouch method. If it returns true, it indicates that the touch event has been processed, and the result will be set to true. If the touch event is not processed by onTouchListener, the onTouchEvent() method of view will be executed. If onTouchEvent() returns true, it means that the touch event is processed by the view, and the result is set to true. It can be seen from the above that in the dispatchTouchEvent() method, the onTouchListener’s ontouch method is executed first. Once it returns true, the view’s ontoucheevent method will not be called. Only when onTouchListener does not handle the touch event will the view’s ontoucheevent method be executed later.

onTouchEvent()
dispatchTouchEvent() is the entry point for the view to handle touch events. In this method, the view will first check whether it has been set onTouchListener. If it has, it will call onTouchListener’s ontouch method. If it returns true, it indicates that the touch event has been processed, and the result will be set to true. If the touch event is not processed by onTouchListener, the onTouchEvent() method of view will be executed. If onTouchEvent() returns true, it means that the touch event is processed by the view, and the result is set to true. It can be seen from the above that in the dispatchTouchEvent() method, the onTouchListener’s onTouch() method is executed first. Once it returns true, the view’s onTouchEvent() method will not be called. Only when onTouchListener does not handle the touch event will the view’s onTouchEvent() method be executed later.

At last, Let’s get a summary. dispatchTouchEvent() is the entry point for the view to handle touch events. In this method, the view will first check whether it has been set onTouchListener. If it has, it will call onTouchListener’s onTouch() method. If it returns true, it indicates that the touch event has been processed, and the result will be set to true. If the touch event is not processed by onTouchListener, the onTouchEvent() method of view will be executed. If onTouchEvent() returns true, it means that the touch event is processed by the view, and the result is set to true. It can be seen from the above that in the dispatchTouchEvent() method, the onTouchListener’s ontouch method is executed first. Once it returns true, the view’s onTouchEvent() method will not be called. Only when onTouchListener does not handle the touch event will the view’s onTouchEvent() method be executed later.

Android 如何上报 Touchevent 给应用层

David Dong — Thu, 10 Jul 2014 23:44:07 +0800

上一篇博文「ATMEL maXTouch IC驱动代码分析」我们讲到了 Touch 驱动代码如何读取IC内部获取到的触摸事件信息并通过input_report_abs()和input_sync()函数上报给 Linux 的 Input 子系统的过程。今天这篇文章我们就走进 Input 子系统内部来看一下事件是如何被传递到 Android 的用户空间的。

内容如下。

1. Input 子系统框架

首先我们从 Input 子系统介绍开始。Input 子系统由驱动层、输入子系统核心层（Input Core）和事件处理层（Event Handler）3部分组成。一个输入事件，如鼠标移动，触摸事件等通过驱动层->系统核心层->事件处理层->用户空间的顺序到达用户空间并传给应用程序使用。其中Input Core即输入子系统核心层由 driver/input/input.c 及相关头文件实现。其对下提供了设备驱动的接口，对上提供了事件处理层的编程接口。输入子系统主要设计input_dev、input_handler、input_handle等数据结构，它们的用途和功能如下图所示。

2. 注册 Input 设备

我们在之前介绍驱动代码的时候讲到过，输入设备在初始化的时候都需要调用input_allocate_device()和input_register_device()进行注册。其中input_allocate_device()函数在内存中为输入设备结构体分配一个空间，并对其主要的成员进行了初始化。它的代码如下。

struct input_dev *input_allocate_device(void)
{
 struct input_dev *dev;
 dev = kzalloc(sizeof(struct input_dev), GFP_KERNEL);
/*分配一个input_dev结构体，并初始化为0*/
 if (dev) {
 	dev->dev.type = &input_dev_type; 	/*初始化设备的类型*/
	dev->dev.class = &input_class; 		/*设置为输入设备类*/
 	device_initialize(&dev->dev); 		/*初始化device结构*/
 	mutex_init(&dev->mutex); 			/*初始化互斥锁*/
 	spin_lock_init(&dev->event_lock); 	/*初始化事件自旋锁*/
 	INIT_LIST_HEAD(&dev->h_list); 		/*初始化链表*/
 	INIT_LIST_HEAD(&dev->node); 		/*初始化链表*/
 	__module_get(THIS_MODULE); 			/*模块引用加1*/
 }
 return dev;
}

该函数返回一个指向input_dev类型的指针，该结构体是一个输入设备结构体，包含了输入设备的一些相关信息，如设备支持的按键码、设备的名称、设备支持的事件等。

接下来调用的input_register_device()函数很重要，我们看一下它的具体实现。

int input_register_device(struct input_dev *dev)
{
	static atomic_t input_no = ATOMIC_INIT(0);
	struct input_handler *handler;
	const char *path;
	int error;
	__set_bit(EV_SYN, dev->evbit);
	init_timer(&dev->timer);
	if (!dev->rep[REP_DELAY] && !dev->rep[REP_PERIOD]) {
		dev->timer.data = (long) dev;
		dev->timer.function = input_repeat_key;
		dev->rep[REP_DELAY] = 250;
		dev->rep[REP_PERIOD] = 33;
	}
	if (!dev->getkeycode)
		dev->getkeycode = input_default_getkeycode;
	if (!dev->setkeycode)
		dev->setkeycode = input_default_setkeycode;
	dev_set_name(&dev->dev, "input%ld", (unsigned long) atomic_inc_return(&input_no) - 1);
	error = device_add(&dev->dev);
	if (error)
		return error;
	path = kobject_get_path(&dev->dev.kobj, GFP_KERNEL);
	printk(KERN_INFO "input: %s as %s\n", dev->name ? dev->name : "Unspecified device", path ? path : "NA");
	kfree(path);
	error = mutex_lock_interruptible(&input_mutex);
	if (error) {
		device_del(&dev->dev);
		return error;
	}
	list_add_tail(&dev->node, &input_dev_list);
	list_for_each_entry(handler, &input_handler_list, node)
	input_attach_handler(dev, handler);
	input_wakeup_procfs_readers();
	mutex_unlock(&input_mutex);
	return 0;
}

input_register_device()函数是输入子系统核心（input core）提供的函数。该函数将input_dev结构体注册到输入子系统核心中，input_dev结构体必须由前面讲的 input_allocate_device()函数来分配。input_register_device()函数如果注册失败，必须调用 input_free_device() 函数释放分配的空间。如果该函数注册成功，在卸载函数中应该调用 input_unregister_device() 函数来注销输入设备结构体。
input_register_device()函数主要完成了如下的工作：

函数中调用__set_bit()函数设置input_dev所支持的事件类型。事件类型由input_dev的evbit成员来表示，在这里将其EV_SYN置位，表示设备支持所有的事件。注意，一个设备可以支持一种或者多种事件类型。常用的事件类型如下：

#define EV_SYN 0x00 /*表示设备支持所有的事件*/
#define EV_KEY 0x01 /*键盘或者按键，表示一个键码*/
#define EV_REL 0x02 /*鼠标设备，表示一个相对的光标位置结果*/
#define EV_ABS 0x03 /*手写板产生的值，其是一个绝对整数值*/
#define EV_MSC 0x04 /*其他类型*/
#define EV_LED 0x11 /*LED灯设备*/
#define EV_SND 0x12 /*蜂鸣器，输入声音*/
#define EV_REP 0x14 /*允许重复按键类型*/
#define EV_PWR 0x16 /*电源管理事件*/

调用dev_set_name()设置input_dev中的device的名字，名字以input0、input1、input2、input3、input4等的形式出现在sysfs文件系统中。
使用device_add()函数将input_dev包含的device结构注册到Linux设备模型中，并可以在sysfs文件系统中表现出来。
调用list_add_tail()函数将input_dev加入input_dev_list链表中，input_dev_list链表中包含了系统中所有的input_dev设备。
调用了input_attach_handler()函数， input_attach_handler()函数用来匹配input_dev和input_handler，只有匹配成功，才能进行下一步的关联操作。
input_attach_handler()函数的代码如下：

static int input_attach_handler(struct input_dev *dev, struct input_handler *handler)
{
	const struct input_device_id *id; /*输入设备的指针*/
	int error;
	if (handler->blacklist && input_match_device(handler->blacklist, dev))
		return -ENODEV; /*设备和处理函数之间的匹配*/
	id = input_match_device(handler->id_table, dev);
	if (!id)
		return -ENODEV;
	error = handler->connect(handler, dev, id);/*连接设备和处理函数*/
	if (error && error != -ENODEV)
		printk(KERN_ERR, "input: failed to attach handler %s to device %s,"error: %d\n",
			handler->name, kobject_name(&dev->dev.kobj), error);
	return error;
}

input_attach_handler()主要完成的工作有：

首先判断handle的blacklist是否被赋值，如果被赋值，则匹配blacklist中的数据跟dev->id的数据是否匹配。blacklist是一个input_device_id*的类型，其指向input_device_ids的一个表，这个表中存放了驱动程序应该忽略的设备。即使在id_table中找到支持的项，也应该忽略这种设备。
调用input_match_device()函数匹配handle->id_table和dev->id中的数据。如果不成功则返回。handle->id_table也是一个input_device_id类型的指针，其表示驱动支持的设备列表。
如果匹配成功，则调用handler->connect()函数将handler与input_dev连接起来。
input_match_device()函数
input_match_device()函数用来与input_dev和handler进行匹配。handler的id_table表中定义了其支持的input_dev设备。
该函数的代码如下：

static const struct input_device_id *input_match_device(const struct
input_device_id *id,
struct input_dev *dev)
{
	int i;
	for (; id->flags || id->driver_info; id++) {
		if (id->flags & INPUT_DEVICE_ID_MATCH_BUS)
			if (id->bustype != dev->id.bustype)
				continue;
		if (id->flags & INPUT_DEVICE_ID_MATCH_VENDOR)
			if (id->vendor != dev->id.vendor)
				continue;
		if (id->flags & INPUT_DEVICE_ID_MATCH_PRODUCT)
			if (id->product != dev->id.product)
				continue;
		if (id->flags & INPUT_DEVICE_ID_MATCH_VERSION)
			if (id->version != dev->id.version)
				continue;
		MATCH_BIT(evbit, EV_MAX);
		MATCH_BIT(keybit, KEY_MAX);
		MATCH_BIT(relbit, REL_MAX);
		MATCH_BIT(absbit, ABS_MAX);
		MATCH_BIT(mscbit, MSC_MAX);
		MATCH_BIT(ledbit, LED_MAX);
		MATCH_BIT(sndbit, SND_MAX);
		MATCH_BIT(ffbit, FF_MAX);
		MATCH_BIT(swbit, SW_MAX);
		return id;
	}
	return NULL;
}

input_match_device()主要完成的工作有：

匹配设备的产品总线类型/vendor/版本信息。
如果id->flags定义的类型匹配成功，或者id->flags没有定义，才会进入到MATCH_BIT的匹配项。 MATCH_BIT宏的定义如下：

#define MATCH_BIT(bit, max) \
for (i = 0; i < BITS_TO_LONGS(max); i++) \
if ((id->bit[i] & dev->bit[i]) != id->bit[i]) \
break; \
if (i != BITS_TO_LONGS(max)) \
continue;

从MATCH_BIT宏的定义可以看出。只有当iput device和input handler的ID成员在evbit、keybit、… swbit项相同才会匹配成功。

3. 数据上报过程

Input 子系统各层之间通信的基本单位就是事件，任何一个输入设备的动作都可以抽像成一种事件，如键盘的按下，触摸屏的按下，鼠标的移动等。事件有三种属性：类型（type），编码(code)，值(value)，Input子系统支持的所有事件都定义在input.h中，包括所有支持的类型，所属类型支持的编码等。事件传送的方向是硬件驱动层–>子系统核心–>事件处理层–>用户空间。

在驱动代码的介绍中，我们讲到驱动最终调用到input_report_abs()将touchevent打包发送给Input子系统。 input_report_abs()函数代码如下：

static inline void input_report_abs(struct input_dev *dev, unsigned int code, int value)
{
    input_event(dev, EV_ABS, code, value);
}

可以看到其为内联函数, 为input_event(,EV_ABS, ...)的二次封装。

input_event()的代码如下,略过无关的部分：

void input_event(struct input_dev *dev,unsigned int type, unsigned int code, int value)
{
	....//event是否支持, 这个和驱动里probe()时填充能力,设置参数有关,略过
    if (is_event_supported(type, dev->evbit, EV_MAX)) {
	....
        input_handle_event(dev, type, code, value);
	...
}

input_event() 调用了input_handle_event()函数。

static void input_handle_event(struct input_dev *dev, unsigned int type, unsigned int code, int value)
{
    int disposition = input_get_disposition(dev, type, code, &value); //得到disposition
......
    if (disposition & INPUT_FLUSH) {
        if (dev->num_vals >= 2)
            input_pass_values(dev, dev->vals, dev->num_vals);
        dev->num_vals = 0;
    } else if (dev->num_vals >= dev->max_vals - 2) {
        dev->vals[dev->num_vals++] = input_value_sync;
        input_pass_values(dev, dev->vals, dev->num_vals);  //**<--> 重点,
        dev->num_vals = 0;
    }

}

所以我们可以简单看下input_handle_event() –> input_get_disposition() EV_SYN事件和EV_ABS的返回值。

static int input_get_disposition(struct input_dev *dev, unsigned int type, unsigned int code, int *pval)
{
    int disposition = INPUT_IGNORE_EVENT;
	......
    switch (type) {

    case EV_SYN:
        switch (code) {
        case SYN_CONFIG:
            disposition = INPUT_PASS_TO_ALL;
            break;

        case SYN_REPORT:
            disposition = INPUT_PASS_TO_HANDLERS | INPUT_FLUSH;
            break;
        case SYN_MT_REPORT:
            disposition = INPUT_PASS_TO_HANDLERS;
            break;
        }
        break;
	......
    case EV_ABS:
        if (is_event_supported(code, dev->absbit, ABS_MAX))
            disposition = input_handle_abs_event(dev, code, &value);//这个可以看看,他会对相同值进行过滤,返回INPUT_IGNORE_EVENT

        break;
	......
    return disposition;
}

让我们回到input_handle_event() –> input_pass_values()

static void input_pass_values(struct input_dev *dev, struct input_value *vals, unsigned int count)
{
	......
    if (handle) {
        count = input_to_handler(handle, vals, count);
    } else {
        list_for_each_entry_rcu(handle, &dev->h_list, d_node)
            if (handle->open) {
                count = input_to_handler(handle, vals, count);
                if (!count)
                    break;
            }
    }
	......
}

其重点函数为input_to_handler()

static unsigned int input_to_handler(struct input_handle *handle,
            struct input_value *vals, unsigned int count)
{
    struct input_handler *handler = handle->handler;
	......
    if (handler->filter) {
        for (v = vals; v != vals + count; v++) {
            if (handler->filter(handle, v->type, v->code, v->value))
                continue;
	.......
    }
	......
    if (handler->events)
        handler->events(handle, vals, count); //<--handler的events.
    else if (handler->event)
        for (v = vals; v != vals + count; v++)
            handler->event(handle, v->type, v->code, v->value);

    return count;
}

这里面input_handle结构体代表一个成功配对的input_dev和input_handler。

关于input_handle，input_dev和input_handler结构体的含义如下：

struct input_dev：物理输入设备的基本数据结构，包含设备相关的一些信息。
struct input_handler：事件处理结构体，定义怎么处理事件的逻辑。
struct input_handle：用来创建input_dev和input_handler之间关系的结构体。

3.1 input_handler

input_handler结构体的定义如下：

struct input_handler {

    void *private;

    void (*event)(struct input_handle *handle, unsigned int type, unsigned int code, int value);
    void (*events)(struct input_handle *handle,
               const struct input_value *vals, unsigned int count);
    bool (*filter)(struct input_handle *handle, unsigned int type, unsigned int code, int value);
    bool (*match)(struct input_handler *handler, struct input_dev *dev);
    int (*connect)(struct input_handler *handler, struct input_dev *dev, const struct input_device_id *id);
    void (*disconnect)(struct input_handle *handle);
    void (*start)(struct input_handle *handle);
......
    const char *name;

    const struct input_device_id *id_table;

    struct list_head    h_list;
    struct list_head    node;
};

该结构体主要是

定义了一个event()处理函数，这个函数将被输入子系统调用去处理发送给设备的事件。例如将发送一个事件命令LED灯点亮，实际控制硬件的点亮操作就可以放在event()函数中实现。
定义了一个connect()函数，该函数用来连接handler和input_dev。
定义了一个disconnect()函数，这个函数用来断开handler和input_dev之间的联系。
定义了一个name，表示handler的名字，显示在/proc/bus/input/handlers目录中。
定义了一个id_table表，表示驱动能够处理的表。
指向一个input_device_id表，这个表包含handler应该忽略的设备。
定义了一个链表h_list，表示与这个input_handler相联系的下一个handler。
定义了一个链表node，将其连接到全局的input_handler_list链表中，所有的input_handler都连接在其上。

3.2 注册 input_handler

input_register_handler()函数注册一个新的input handler处理器。这个handler将为输入设备使用，一个handler可以添加到多个支持它的设备中，也就是一个handler可以处理多个输入设备的事件。函数的参数传入简要注册的input_handler指针，该函数的代码如下：

int input_register_handler(struct input_handler *handler)
{
......//初始化h_list
    INIT_LIST_HEAD(&handler->h_list);
    //将node加到list尾部
    list_add_tail(&handler->node, &input_handler_list);
    //在注册handler的时候也对已有设备调用一次attach()
    list_for_each_entry(dev, &input_dev_list, node)
        input_attach_handler(dev, handler);
......
}

完成的主要工作：

调用list_add_tail()函数，将handler加入全局的input_handler_list链表中，该链表包含了系统中所有的input_handler。
调用了input_attach_handler()函数。input_attach_handler()函数的作用是匹配 input_dev_list链表中的input_dev与handler。如果成功会将input_dev与handler联系起来。

3.3 input_handle

Input_Handle 结构体
input_register_handle()函数用来注册一个新的handle到输入子系统中。input_handle的主要功能是用来连接input_dev和input_handler。

struct input_handle {
	void *private;
	int open;
	const char *name;
	struct input_dev *dev;
	struct input_handler *handler;
	struct list_head d_node;
	struct list_head h_node;
};

3.4 注册 input_handle

input_handle是用来连接input_dev和input_handler的一个中间结构体。事件通过input_handle从 input_dev 发送到input_handler，或者从input_handler发送到input_dev进行处理。在使用input_handle之前，需要对其进行注册，注册函数是input_register_handle()。 input_register_handle()函数用来注册一个新的handle到输入子系统中。该函数接收一个input_handle类型的指针，该变量要在注册前对其成员初始化。
input_register_handle()函数的代码如下：

int input_register_handle(struct input_handle *handle)
{
	struct input_handler *handlehandler = handle->handler;
	struct input_dev *dev = handle->dev;
	int error;
	error = mutex_lock_interruptible(&dev->mutex);
	if (error)
		return error;
	list_add_tail_rcu(&handle->d_node, &dev->h_list);
	mutex_unlock(&dev->mutex);
	synchronize_rcu();
	list_add_tail(&handle->h_node, &handler->h_list);
	if (handler->start)
		handler->start(handle);
	return 0;
}

调用list_add_tail_rcu()函数将handle加入输入设备的dev->h_list链表中。
调用list_add_tail()函数将handle加入input_handler的handler->h_list链表中。

input_dev、input_handler和handle三者之间的关系如下：

input_dev 是硬件驱动层，代表一个input设备；
input_handler 是事件处理层，代表一个事件处理器；
input_handle 个人认为属于核心层，代表一个配对的input设备与input事件处理器；
input_dev 通过全局的input_dev_list链接在一起。设备注册的时候实现这个操作；
input_handler 通过全局的input_handler_list链接在一起。事件处理器注册的时候实现这个操作（事件处理器一般内核自带，一般不需要我们来写）；
input_hande 没有一个全局的链表，它注册的时候将自己分别挂在了input_dev 和input_handler 的h_list上了。通过input_dev 和input_handler就可以找到input_handle 在设备注册和事件处理器，注册的时候都要进行配对工作，配对后就会实现链接。通过input_handle也可以找到input_dev和input_handler。

3.5 由核心层 (inputcore) 到事件处理层 (eventhandler)

我们看到上面的代码调用到

handler->events(handle, v->type, v->code, v->value);

这里handler->events则是Evdev.c(drivers\input)里定义的。 events 函数是当事件处理器接收到了来自input设备传来的事件时调用的处理函数，负责处理事件。

static struct input_handler evdev_handler = {

	.event = evdev_event,
	.events = evdev_events,
	...
}

我们看一下函数原型。

static void evdev_events(struct input_handle *handle,
             const struct input_value *vals, unsigned int count)
{
......
    if (client)
        evdev_pass_values(client, vals, count, ev_time);
    else
        list_for_each_entry_rcu(client, &evdev->client_list, node)
            evdev_pass_values(client, vals, count, ev_time);
......
}

事件处理层（eventhandler）负责将事件上报，将键值、坐标等数据上报的对应的设备节点.

static void evdev_pass_values(struct evdev_client *client,
            const struct input_value *vals, unsigned int count,
            ktime_t *ev_time)
{
    struct evdev *evdev = client->evdev;
    const struct input_value *v;
    struct input_event event;
    struct timespec64 ts;
......//时间
    event.input_event_sec = ts.tv_sec;
    event.input_event_usec = ts.tv_nsec / NSEC_PER_USEC;
......
    for (v = vals; v != vals + count; v++) {
......//事件数据填充
        event.type = v->type;
        event.code = v->code;
        event.value = v->value;
        __pass_event(client, &event); //<--放到client->buffer里
    }
......
}

3.6 由事件处理层 (eventhandler) 到用户空间（user space）

__pass_event()将event放到client->buffer[]里,由buffer 传入用户空间。
__pass_event() 函数最终将事件传递给了用户端的client 结构中的input_event 数组中，只需将这个input_event数组复制给用户空间，进程就能收到触摸屏按下的信息了。

static void __pass_event(struct evdev_client *client,
             const struct input_event *event)
{
    client->buffer[client->head++] = *event;
    client->head &= client->bufsize - 1;

    if (unlikely(client->head == client->tail)) {
        /*
         * This effectively "drops" all unconsumed events, leaving
         * EV_SYN/SYN_DROPPED plus the newest event in the queue.
         */
        client->tail = (client->head - 2) & (client->bufsize - 1);

        client->buffer[client->tail].input_event_sec =
                        event->input_event_sec;
        client->buffer[client->tail].input_event_usec =
                        event->input_event_usec;
        client->buffer[client->tail].type = EV_SYN;
        client->buffer[client->tail].code = SYN_DROPPED;
        client->buffer[client->tail].value = 0;

        client->packet_head = client->tail;
    }

    if (event->type == EV_SYN && event->code == SYN_REPORT) {
        client->packet_head = client->head;
        kill_fasync(&client->fasync, SIGIO, POLL_IN);
    }
}

input_event 结构体：

struct input_event
{
	struct timeval time;
	__u16 type;
	__u16 code;
	__s32 value;
}

3.7 用户空间读取事件

我们从上面分析,看到数据已经放到了client->buffer[], 那读取也肯定也是从这里读。实际上，在文件evdev.c 中Evdev_read()函数将这个input_event数组复制给用户空间。

static ssize_t evdev_read(struct file *file, char __user *buffer,
              size_t count, loff_t *ppos)
{
    struct evdev_client *client = file->private_data;
    struct evdev *evdev = client->evdev;
......
    for (;;) {
......//循环读取下一个事件, 并通过input_event_to_user() --> copy_to_user()给用户空间, 这样上面就读到数据了.
        while (read + input_event_size() <= count &&
               evdev_fetch_next_event(client, &event)) {

            if (input_event_to_user(buffer + read, &event))
......
    return read;
}

调用了input_event_to_user()函数

int input_event_to_user(char __user *buffer, const struct input_event * event){
	
	if(copy_to_user(buffer,event,sizeof(struct input_event)))
		return -EFAULT;

	return 0;
}

事件读取函数调用流程

read时候 evdev_read--> 从client->buffer[]循环获取事件 evdev_fetch_next_event() --> input_event_to_user() --> copy_to_user()

4. 总结

最后总结一下整个数据的走向和传送的流程。

按照linux设备架构,驱动模型实现touchscreen driver。
模块初始化函数中将触摸屏注册到了输入子系统中，于此同时，注册函数在事件处理层链表中寻找事件处理器，这里找到的是evdev，并且将驱动与事件处理器挂载。并且在/dev/input中生成设备文件event0，以后我们访问这个文件就会访问到设备数据。

当各个handler init时 --> input_register_handler() --> input_attach_handler() -->  handler->connect()
或者驱动 --> probe() --> input_register_device() --> input_attach_handler -->  handler->connect()

                                             +--> input_register_handle() dev和handler关联
handler->connect()-->  eg:evdev.c events() --+
                                             +-->cdev_device_add() 注册字符设备

当点击触屏后, 进到中断处理,然后读取数据,再report,并存到client的buffer[]里。

input_report_abs() --> input_event(, EV_ABS, , ) --> input_handle_event() --> input_pass_values() --> input_to_handler() -->
handler->events()/event() --> eg:evdev.c events() --> evdev_pass_values() --> 数据填充 --> __pass_event() --> client->buffer[]

上层用户空调read时, 只要有数据,不断从client->buffer[]读取并通过copy_to_user()拷到用户空间, 所以上层就拿到数据了。

read时候...--> evdev_read--> 从client->buffer[]循环获取事件 evdev_fetch_next_event() --> 
input_event_to_user() --> copy_to_user()

ATMEL maXTouch IC 驱动代码分析

David Dong — Wed, 25 Jun 2014 17:18:35 +0800

今天来讲一下 touch controller IC 在 android 上的驱动代码。

Touch Screen 作为一个 input device, 驱动代码当然要符合 android 对输入设备的一般性要求。

实际上驱动代码完成的工作相对简单，主要有以下这么几个内容。

初始化设备
完成 Input device 在 andorid 的注册
注册中断
初始化相关的系统文件节点
上报 input event 给上层 EventHub
提供一些调试接口

下面就以 ATMEL maXTouch IC 的驱动代码为例，分析相关的实现过程。源码请参考 GitHub 的项目主页「github 源码」

1. 设备初始化

首先 Touch IC 是一个 I2C 的设备，因此需要在内核里注册 I2C 的设备并和驱动代码匹配。有关内核搜索设备驱动并和注册设备匹配的内容可以去参考相关的文档，这里需要注意的是 i2c_device_id 里的 ID 名称一定要和设备树里面注册的 ID 名称一致。才能保证内核会加载到正确的驱动代码。

static const struct i2c_device_id mxt_id[] = {
    { "qt602240_ts", 0 },
    { "atmel_mxt_ts", 0 },
    { "mXT224", 0 },
    { }
};
MODULE_DEVICE_TABLE(i2c, mxt_id);

static struct i2c_driver mxt_driver = {
    .driver = {
        .name    = "atmel_mxt_ts",
        .owner    = THIS_MODULE,
#ifdef CONFIG_PM
        .pm    = &mxt_pm_ops,
#endif
    },
    .probe        = mxt_probe,
    .remove        = __devexit_p(mxt_remove),
    .id_table    = mxt_id,
};

static int __init mxt_init(void)
{
    return i2c_add_driver(&mxt_driver);
}

static void __exit mxt_exit(void)
{
    i2c_del_driver(&mxt_driver);
}

module_init(mxt_init);
module_exit(mxt_exit);

内核在加载到该设备的驱动后会执行 Probe() 函数对设备进行初始化。在 probe() 函数中驱动主要完成的内容有

static int __devinit mxt_probe(struct i2c_client *client,
        const struct i2c_device_id *id)
{
    const struct mxt_platform_data *pdata = client->dev.platform_data;
    struct mxt_data *data;
    struct input_dev *input_dev;
    int error;

    if (!pdata)
        return -EINVAL;
    
    data = kzalloc(sizeof(struct mxt_data), GFP_KERNEL);
    input_dev = input_allocate_device();
    if (!data || !input_dev) {
        dev_err(&client->dev, "Failed to allocate memory\n");
        error = -ENOMEM;
        goto err_free_mem;
    }

这里面结构体 input_dev 用于描述一个输入子系统设备，任何驱动设备如果想标明自己是输入设备，都应该通过初始化这样的结构体。input_allocate_device()这个函数会为 input_dev 这个结构体申请内存并完成这个结构体在内核中的注册。关于这个函数的说明请参考我的另一篇文章「Android 如何上报 Touchevent 给应用层」。

/* Initialize i2c device */
    error = mxt_initialize(data);

接下来要执行 mxt_initialize() 这个函数来做设备的硬件初始化。我们来看看硬件初始化里面都做了哪些事情。

static int mxt_initialize(struct mxt_data *data)
{
    struct i2c_client *client = data->client;
    struct mxt_info *info = &data->info;
    int error;

    error = mxt_get_info(data);
    if (error) {
        error = mxt_probe_bootloader(data);
    
        if (error) {
            return error;
        } else {
            data->state = BOOTLOADER;
            return 0;
        }
    }
    
    dev_info(&client->dev,
        "Family ID: %d Variant ID: %d Version: %d.%d "
        "Build: 0x%02X Object Num: %d\n",
        info->family_id, info->variant_id,
        info->version >> 4, info->version & 0xf,
        info->build, info->object_num);
    
    data->state = APPMODE;
    
    data->object_table = kcalloc(info->object_num,
                     sizeof(struct mxt_object),
                     GFP_KERNEL);
    if (!data->object_table) {
        dev_err(&client->dev, "Failed to allocate memory\n");
        return -ENOMEM;
    }
    
    /* Get object table information */
    error = mxt_get_object_table(data);
    if (error) {
        dev_err(&client->dev, "Error %d reading object table\n", error);
        return error;
    }
    
    error = mxt_check_message_length(data);
    if (error)
        return error;
    
    error = mxt_probe_power_cfg(data);
    if (error) {
        dev_err(&client->dev, "Failed to initialize power cfg\n");
        return error;
    }
    
    /* Check register init values */
    error = mxt_check_reg_init(data);
    if (error) {
        dev_err(&client->dev, "Failed to initialize config\n");
        return error;
    }
    
    error = mxt_read_resolution(data);
    if (error) {
        dev_err(&client->dev, "Failed to initialize screen size\n");
        return error;
    }
    
    return 0;

}

我们看到首先要从芯片内部读取相关的设备信息(芯片型号，版本信息等)，这些信息是存储在芯片内部 memeory 的固定的地址。调用的函数 mxt_get_info()，该函数的返回值表示读取的结果，如果不为零说明读取失败，芯片状态异常，这时要通过发送命令让芯片进入 bootloader 模式(mxt_probe_bootloader()函数)，重新复位。

error = mxt_get_info(data);

如果读取正确，设置芯片状态为 APP_MODE，这时会读取 object_table，ATMEL 的 Touch IC 内部维护了一个寄存器的地址列表，不同型号的 IC，该地址列表内容有所不同。通过读取该列表来初始化寄存器的地址。从而可以正确读写内部寄存器。具体的内容需要参考芯片的技术手册。

error = mxt_get_object_table(data);

拿到寄存器的地址后，要读取一些寄存器的值来获取相关的状态（寄存器在芯片出厂厂测时会被写入初始的值）。

error = mxt_check_message_length(data);
    if (error)
        return error;

    error = mxt_probe_power_cfg(data);
    if (error) {
        dev_err(&client->dev, "Failed to initialize power cfg\n");
        return error;
    }
    
    /* Check register init values */
    error = mxt_check_reg_init(data);
    if (error) {
        dev_err(&client->dev, "Failed to initialize config\n");
        return error;
    }

最后一步是读取寄存器内部配置的显示屏的分辨率信息，为以后上报 touch event 的坐标数据做准备。

error = mxt_read_resolution(data);

到这里，刚才提到的第一步针对芯片硬件的初始化就完成了。接下来要完成

完成 Input device 在 andorid 的注册
注册中断
初始化相关的文件节点

2. 注册 Input Device

/* Initialize input device */
    input_dev->name = "Atmel maXTouch Touchscreen";
    input_dev->id.bustype = BUS_I2C;
    input_dev->dev.parent = &client->dev;
    input_dev->open = mxt_input_open;
    input_dev->close = mxt_input_close;

    __set_bit(EV_ABS, input_dev->evbit);
    __set_bit(EV_KEY, input_dev->evbit);
    __set_bit(BTN_TOUCH, input_dev->keybit);
    
    /* For single touch */
    input_set_abs_params(input_dev, ABS_X,
                 0, data->max_x, 0, 0);
    input_set_abs_params(input_dev, ABS_Y,
                 0, data->max_y, 0, 0);
    input_set_abs_params(input_dev, ABS_PRESSURE,
                 0, 255, 0, 0);
    
    /* For multi touch */
    input_mt_init_slots(input_dev, MXT_MAX_FINGER);
    input_set_abs_params(input_dev, ABS_MT_TOUCH_MAJOR,
                 0, MXT_MAX_AREA, 0, 0);
    input_set_abs_params(input_dev, ABS_MT_POSITION_X,
                 0, data->max_x, 0, 0);
    input_set_abs_params(input_dev, ABS_MT_POSITION_Y,
                 0, data->max_y, 0, 0);
    input_set_abs_params(input_dev, ABS_MT_PRESSURE,
                 0, 255, 0, 0);
    
    input_set_drvdata(input_dev, data);
    
    error = input_register_device(input_dev);
    if (error) {
        dev_err(&client->dev, "Error %d registering input device\n",
            error);
        goto err_free_irq;
    }

以上为注册 Input device 的代码，这里涉及到 Linux input 设备的初始化，需要调用 __set_bit(), input_set_abs_params() 函数来完成输入设备的一些必要的配置，比如 Input 事件类型，多少个手指，分辨率是多少等，具体可以参考「Android 如何上报 Touchevent 给应用层」。
最后调用input_register_device()函数来将刚才配置好的 Input device 注册到 kernel 中去。

3. 初始化中断

关于申请中断就比较简单，需要调用request_threaded_irq()函数，该函数的参数中需要传入

该设备申请的中断号，这个在 DTS 中定义。
中断处理函数名
中断的类型
驱动的名字

error = request_threaded_irq(client->irq, NULL, mxt_interrupt,
            pdata->irqflags, client->dev.driver->name, data);
    if (error) {
        dev_err(&client->dev, "Error %d registering irq\n", error);
        goto err_free_object;
    }

4. 注册 Sys 文件节点

最后的步骤是注册 sys 文件节点，后期通过读写这些文件节点可以完成对芯片的特定操作，比如升级固件，配置文件等。

error = sysfs_create_group(&client->dev.kobj, &mxt_attr_group);
    if (error) {
        dev_err(&client->dev, "Failure %d creating sysfs group\n",
            error);
        goto err_unregister_device;
    }

sysfs_create_group()这个函数需要传入一个 attribute_group 结构体的地址。这个地址实际上指向了一个文件属性 attribute 类型的指针数组，通过该数组可以引用到文件节点操作的函数。

static struct attribute *mxt_attrs[] = {
    &dev_attr_update_fw.attr,
    &dev_attr_debug_enable.attr,
    &dev_attr_pause_driver.attr,
    NULL
};

至此，驱动的加载已经完成，该驱动支持的设备已经可以正常使用。当然驱动代码还有其他的一些任务，比如定义系统休眠，唤醒时的操作。实际上就是设备上下电相关的一些操作。如果需要更改设备上下电时的策略，则要对 mxt_start() 和 mxt_stop() 两个函数内容进行修改。

5. 设备上下电

static int mxt_suspend(struct device *dev)
{
    struct i2c_client *client = to_i2c_client(dev);
    struct mxt_data *data = i2c_get_clientdata(client);
    struct input_dev *input_dev = data->input_dev;

    mutex_lock(&input_dev->mutex);
    
    if (input_dev->users)
        mxt_stop(data);
    
    mutex_unlock(&input_dev->mutex);
    
    return 0;

}

static int mxt_resume(struct device *dev)
{
    struct i2c_client *client = to_i2c_client(dev);
    struct mxt_data *data = i2c_get_clientdata(client);
    struct input_dev *input_dev = data->input_dev;

    /* Soft reset */
    mxt_soft_reset(data, MXT_RESET_VALUE);
    
    mutex_lock(&input_dev->mutex);
    
    if (input_dev->users)
        mxt_start(data);
    
    mutex_unlock(&input_dev->mutex);
    
    return 0;

}

6. 中断处理程序

最后也是最重要的就是中断处理程序了，驱动代码要在中断处理程序中将触摸事件转成 input_event_message 类型的数据帧，发给内核的 EVENT_HUB 来处理。

static irqreturn_t mxt_interrupt(int irq, void *dev_id)
{
    struct mxt_data *data = dev_id;
    struct mxt_message message;
    struct mxt_object *object;
    struct device *dev = &data->client->dev;
    int touchid;
    u8 reportid;

    do {
        if (mxt_read_message(data, &message)) {
            dev_err(dev, "Failed to read message\n");
            goto end;
        }
    
        reportid = message.reportid;
    
        object = mxt_get_object(data, MXT_TOUCH_MULTI_T9);
        if (!object)
            goto end;
    
        if (reportid >= object->min_reportid
            && reportid <= object->max_reportid) {
            touchid = reportid - object->min_reportid;
            mxt_input_touchevent(data, &message, touchid);
        } else {
            object = mxt_get_object(data, MXT_GEN_COMMAND_T6);
            if (!object)
                goto end;
    
            if ((reportid == object->max_reportid)
                && (message.message[0] & MXT_STATUS_CFGERROR))
                dev_err(dev, "Configuration error\n");
        }
    } while (reportid != MXT_RPTID_NOMSG);

end:
    return IRQ_HANDLED;
}

可以看到，在中断处理函数中使用了轮询的方法，通过 mxt_read_message() 函数来读取IC准备好的数据，直到所有数据都被读取，然后调用 mxt_input_touchevent() 函数将读取的数据打包发送。 mxt_input_touchevent() 函数的实现如下。

static void mxt_input_touchevent(struct mxt_data *data,
                      struct mxt_message *message, int id)
{
    struct mxt_finger *finger = data->finger;
    struct device *dev = &data->client->dev;
    u8 status = message->message[0];
    int x;
    int y;
    int area;
    int pressure;

    if (data->driver_paused)
        return;
    
    if (id > MXT_MAX_FINGER) {
        dev_err(dev, "MXT_MAX_FINGER exceeded!\n");
        return;
    }
    
    /* Check the touch is present on the screen */
    if (!(status & MXT_DETECT)) {
        if (status & MXT_SUPPRESS) {
            dev_dbg(dev, "[%d] suppressed\n", id);
    
            finger[id].status = MXT_RELEASE;
            mxt_input_report(data, id);
        } else if (status & MXT_RELEASE) {
            dev_dbg(dev, "[%d] released\n", id);
    
            finger[id].status = MXT_RELEASE;
            mxt_input_report(data, id);
        }
        return;
    }
    
    /* Check only AMP detection */
    if (!(status & (MXT_PRESS | MXT_MOVE)))
        return;
    
    x = (message->message[1] << 4) | ((message->message[3] >> 4) & 0xf);
    y = (message->message[2] << 4) | ((message->message[3] & 0xf));
    if (data->max_x < 1024)
        x = x >> 2;
    if (data->max_y < 1024)
        y = y >> 2;
    
    area = message->message[4];
    pressure = message->message[5];
    
    dev_dbg(dev, "[%d] %s x: %d, y: %d, area: %d\n", id,
        status & MXT_MOVE ? "moved" : "pressed",
        x, y, area);
    
    finger[id].status = status & MXT_MOVE ?
                MXT_MOVE : MXT_PRESS;
    finger[id].x = x;
    finger[id].y = y;
    finger[id].area = area;
    finger[id].pressure = pressure;
    
    mxt_input_report(data, id);

}

mxt_input_touchevent() 是十分重要的函数，在这个函数里会根据读取到的数据判断当前 touch 的状态，比如手指是抬起，抑制还是按压，移动。针对不同的状态会发送不同的消息类型给上层。具体的传送通过 mxt_input_report() 函数执行。 mxt_input_report()的函数体如下。

static void mxt_input_report(struct mxt_data *data, int single_id)
{
    struct mxt_finger *finger = data->finger;
    struct input_dev *input_dev = data->input_dev;
    int status = finger[single_id].status;
    int finger_num = 0;
    int id;

    for (id = 0; id < MXT_MAX_FINGER; id++) {
        if (!finger[id].status)
            continue;
    
        input_mt_slot(input_dev, id);
        input_mt_report_slot_state(input_dev, MT_TOOL_FINGER,
                finger[id].status != MXT_RELEASE);
    
        if (finger[id].status != MXT_RELEASE) {
            finger_num++;
            input_report_abs(input_dev, ABS_MT_TOUCH_MAJOR,
                    finger[id].area);
            input_report_abs(input_dev, ABS_MT_POSITION_X,
                    finger[id].x);
            input_report_abs(input_dev, ABS_MT_POSITION_Y,
                    finger[id].y);
            input_report_abs(input_dev, ABS_MT_PRESSURE,
                    finger[id].pressure);
        } else {
            finger[id].status = 0;
        }
    }
    
    input_report_key(input_dev, BTN_TOUCH, finger_num > 0);
    
    if (status != MXT_RELEASE) {
        input_report_abs(input_dev, ABS_X, finger[single_id].x);
        input_report_abs(input_dev, ABS_Y, finger[single_id].y);
        input_report_abs(input_dev,
                 ABS_PRESSURE, finger[single_id].pressure);
    }
    
    input_sync(input_dev);

}

mxt_input_report() 主要调用了如下的 Linux kernel 系统服务来上报消息。

input_mt_slot()
input_mt_report_slot_state()
input_report_abs()
input_report_key()
input_sync()

其中 input_mt_slot() 是指明当前上报的 slot 号，目前的代码使用的是 protocol B 协议来处理 touch 事件，protocol B 会为每个手指分配一个 slot，不同手指的数据会被封装到不同的 slot 中，这样可以保证不同的手指消息被区分开来传送。可以更好的支持多指触控(不同于协议A，触控 IC 的 firmware 可以计算划分不同的手指信息，无需上层的算法参与，可以提高响应速度)，详细的内容可以参考「Linux 下的多点触摸协议」。 input_mt_report_slot_state()是设定当前slot的状态，比如按下，抬起等。如果是按下状态，还要调用input_report_abs()函数来上报当前的坐标信息。如果是按键事件，调用input_report_key()来上报当前的按键信息。最后input_sync()来将所有的信息打包成一个数据帧来发送，注意如果不执行这个函数，之前的信息无效，不会被发送给上层。

好了，到这里整个驱动代码所要完成的主要任务都已经完成了。从整个流程来看虽然比较简单，但是触控IC 的驱动程序作为硬件设备和 Linux kernel 的接口，还是起到了非常重要的作用。对于设备的驱动代码，还是要十分重视。

Bundle 在 android 中的应用

David Dong — Mon, 14 Apr 2014 21:03:36 +0800

Android中Bundle类的作用，根据google 官方文档
Bundle类用作携带数据，它类似于Map，用于存放key-value名值对形式的值。

Bundle经常使用在Activity之间或者线程间传递数据，传递的数据可以是 boolean、byte、int、long、float、double、string 等基本类型或它们对应的数组，也可以是对象或对象数组。当Bundle传递的是对象或对象数组时，必须实现Serializable 或Parcelable接口。 Bundle提供了各种常用类型的putXxx()/getXxx()方法，用于读写基本类型的数据。（各种方法可以查看API）
参考代码如下：

//Student 类应该实现Serializable接口
Student student = new student();

//声明一个Bundle的对象
Bundle bundle = new Bundle();

//将student对象装入bundle中，同时声明key值
bundle.putSerializable("student",student);

//声明一个Intent 对象
Intent intent = new Intent(MainActivity.this,DisplayActivity.class);
//intent.setAction("android.intent.action.DISPLAY");

//通过intent发送bundle到目标activity中，实现activity之间的对象传递
intent.putExtras(bundle);

startActivityForResult(intent,2);

电容式触摸检测技术原理

David Dong — Fri, 11 Apr 2014 12:10:27 +0800

智能手机的兴起不仅仅是由于操作系统上的改变，更多的是由于引入了一种新的人机交互的模式。这种全新的模式导致了用户体验的大幅度改善。电容式触摸检测技术正是其中的关键技术。

电容式触摸检测本质上是一种精密的电信号测量技术，其测量的电荷变化量为皮库伦级别。触控IC通过高精度的ADC采样触摸屏上的电压信号，在人手或者其他导电体接触到设备后，检测到的电压信号会发生变化（mv级别），这些变化量被采样到触控IC内部，从而可以被识别到。

而这其中又涉及到触摸屏ITO的工艺制造和图案的设计，不同的ITO叠层和图案的设计都会对触控检测产生影响。

ITO全称是纳米铟锡金属氧化物，主要有这两点特性，透明和导电。适合应用于在屏幕上贴合透明的导电层的需求。

目前常见的ITO叠层分为双层（搭桥/不搭桥）和单层方案，按照基材来看的话有PET和Glass的选择，按照贴合方式的话分为全贴合和框贴的方式。具体可以演化为 GFF(PFF)/OGS(G1/G2)/GG/GF 等，由于不同的叠层设计导致工艺的难度和良率不同，生产出的触摸屏厚度也不一样，因此大多数手机ODM都会根据不同的ID设计要求和造价来进行评估。在触控的性能方面，一般来说双层ITO由于出线的方式和Cm(X/Y 耦合电容)控制方面的优势，性能上比较容易把握。早期的单层方案适合做单点触控（比如应用三角形图案），目前也有单层多点的方案，但是由于单层出线，在减小bonding pad面积和走线Gap方面都面临一些挑战。

目前也有面板厂商主推的On-Cell/In-Cell 方案，将ITO嵌入到显示面板内部，减少了整体的厚度。现在看来是未来的发展方向，但是技术难度大，良率的控制是一个问题，目前仍有许多的挑战。

ITO的图形常见的有菱形/三角形/长条形/毛毛虫形等。以及基于菱形的一些变形图案，比如雪花形等。其中菱形/长条形应用于两层ITO的设计，三角形应用于单层单点的触控，毛毛虫的图案用于单层多点的开发。性能方面也有不同，比如菱形的方案由于电场的分布一致性好，因此在触摸检测的精度和线性度方面有优势，另外变形的方案，比如雪花形，是在菱形的方案上增加了X,Y层的耦合面积，因此在灵敏度方面更好，另外对于一些小面积的物体，比如1mm的passive stylus的检测精度上有提升。

本文主要是对触控IC检测原理的说明，因此以上关于触摸屏设计方面的内容不会占用太多的篇幅描述，我会另外写一篇博客来展开详细的讨论。

检测原理

我们现在以下面的图片作为开始

电路模型

这张图片代表了一个人机操作的模型，其中手持设备的信号地通过一个耦合电容Cgnd-earth与大地相连。我们将重点放到手持设备上。细化一下模型，如下图。

这是一个完整触摸设备的模型，其中X，Y 分别是触摸屏的ITO 走线X和Y, Cx-y为x和y之间的耦合电容，通常也叫互电容Cm。X线作为信号驱动线连接到触控IC的TX输出端，Y线作为接收线连接到触控IC的输入端口，该端口的信号会被送入IC内部的ADC采样。在ADC的输入端与IC的信号地之间有一个采样电容 Cs 用于积累电荷，在ADC的采样端形成电压信号。注意此图片所示是没有人手触摸的情形下的模型。

我们继续简化一下模型，将它转化为电路的形式。

从以上图就比较清晰了，从IC 信号源正端输出的电荷流经Cx-y和Cs，并最终通过GND 回到信号源的负端。这期间会在Cx-y和Cm两端建立起电压。

我们来对整个过程做一个计算，假设信号源输出的电压为6.6V，Cxy为2pF,Cs为18pF,电流积分电路静态时可以视为短路。ADC 为10位转换，参考电压3.3v。

根据公式

Q = C × ΔV （Qtotal = Csystem × ΔV）

可以计算出系统中的总电荷量为

Csystem = 1/(1/Cx-y + 1/Cs) = 1/(1/2pF + 1/18pF) = 1.8pF

Qtotal = 1.8pF × 6.6V = 11.88pC

其中在Cs两侧的建立的电压为

ΔVacross Cs = Q / Cs = 11.88pC / 18pF = 0.66V。

这个电压会被ADC采样从而得到一个ADC的数值给到IC做计算

0.66V / 3.3V × 1024 = 205 ADC counts

至此我们做一个总结。

电荷	Cs 电压	ADC 值
11.88pc	0.66v	205

下面我们对有手指触摸的情况做一个分析。手指触摸时的模型如下图

由于人体本身是导体，因此在手接触到触摸屏表面后会导入Cx-human和Cy-human两个耦合电容。这就给之前讲到的电荷流经的途径中增加了一个通路，即一部分电荷通过人体流经大地回流到设备信号源。同刚才的例子一样，我们进一步简化模型以电路的形式呈现。

我们依旧继续刚才的计算，假设Cy-human = 2pF, Cx-human = 2pF, Cx-y = 2pF and Cs = 18pF. 则流程系统的总电荷为

Ca = Cy-human + Cs = 2pF + 18pF = 20pF

Cb = 1 / (1/Cx-y + 1/Ca) = 1 / (1/2pF + 1/20pF) = 1.81pF

Csystem = Cx-human + Cb = 2pF + 1.81pF = 3.81pF

Qtotal = Csystem × ΔV = 3.81pF × 6.6V = 25.2pC

我们可以计算出电路中电荷的分布如下

因此可以得出Cs两端的电压

ΔVacross Cs = Q / Cs = 10.8pC / 18pF = 0.6V

而ADC的数值为 0.6V / 3.3V × 1024 = 186 ADC counts

我们把手指触摸时的数据填入刚才的表格，

电荷	Cs 电压	ADC 值
11.88pc	0.66v	205
10.8pc	0.6v	186

我们得出不同情形下两次ADC采样的差值 205-186 = 19

通过两次采样时的ADC差值，触控IC 就可以检测到触摸屏上电压的变化，从而感知到人手触摸的动作。当然以上只是一个简单的模型和处理，用于描述触摸技术的检测原理，实际的应用中会对检测到的信号做很多处理，比如放大，过采样技术，双边沿积分等手段来放大差异，增加信号的robust和可识别性等。以上会涉及到IC厂家的专利保密技术，不在本文的讨论范围内。

Linux 系统下的多点触摸协议

David Dong — Wed, 12 Feb 2014 21:33:10 +0800

这篇文章讲一下Linux下的多点触控的协议。

为了支持多点触控的设备，Linux制定了多点触摸协议（multi-touch，MT）。

Linux 下的触摸协议分为协议A(protocol A)和协议B(protocol B)。

作为都支持多点触摸的协议，那么协议A和协议B有什么区别？我们以下的篇幅里会做详细的说明。

协议A

主要应用于早期处理触点的设备，该类设备无法跟踪并识别每个触摸点，因此协议A里无法区分上报的信息都是由哪些触摸点触发的。内核驱动应该根据设备表面上全部有效触控进行列举并生成事件。每个触控点数据包在这次事件流中的顺序并不重要。事件过滤和手指跟踪的工作留给用户空间来实现。内核的代码上主要是用input_mt_sync()函数在每个数据包的结尾对多个触控包进行分割，每一个input_mt_sync()都在内核引入一个SYN_MT_REPORT事件。这个事件通常表示该组数据包的结束。input_sync()函数表示这次传送数据帧的结束（会有多包数据），上层接收到input_sync()的通知后，会将之前接收的多包数据（由SYN_MT_REPORT分割）一起处理。用户层在接收到全部的数据后，根据和先前同步的触点的数据对比，运行相关的算法来进行数据触摸点的识别和追踪。
一个典型的协议A的数据传输过程如下。

    ABS_MT_POSITION_X x[0]
    ABS_MT_POSITION_Y y[0]
    SYN_MT_REPORT
    ABS_MT_POSITION_X x[1]
    ABS_MT_POSITION_Y y[1]
    SYN_MT_REPORT
    SYN_REPORT

其中SYN_MT_REPORT表示一包数据的结束，SYN_REPORT代表这次传输的结束，在SYN_REPORT之前的数据会被上层统一处理。

协议B

协议B针对的设备有能力鉴别并追踪触摸点。因此在上报每个点的坐标信息以外会附带上触摸点的信息（例如代表是由哪个手指触摸）。比如ATMEL mxTouch IC 的上报信息格式。

这就要求协议B里需要能够上报触摸点的信息，并能够根据不同的触摸点来打包发送数据。这样做的好处是大量的工作由底层的硬件来完成，无需上层运行算法来过滤和追踪触摸点的轨迹。与协议A不同，协议B在每个数据包的开头以slot为参数调用input_mt_slot()来分离触点数据包。这会产生一个ABS_MT_SLOT事件，从而通知接收者准备所给的slot的更新。而在传送的结尾和协议A相同，都是由input_sync()来通知本次传送的结束。一次协议B的数据传送过程如下。

   ABS_MT_SLOT 0
   ABS_MT_TRACKING_ID 45
   ABS_MT_POSITION_X x[0]
   ABS_MT_POSITION_Y y[0]
   ABS_MT_SLOT 1
   ABS_MT_TRACKING_ID 46
   ABS_MT_POSITION_X x[1]
   ABS_MT_POSITION_Y y[1]
   SYN_REPORT

可以看到这次传输有两个触摸点的数据，每包数据以ABS_MT_SLOT起始，每个slot都会被分配一个ABS_MT_TRACKING_ID，不同的可识别的触摸点被打包成不同的slot数据。利用这个slot来传递对应触点的变化，触点的创建、替换和销毁是通过修改相关联slot的ABS_MT_TRACTKING_ID完成的。一个无符号的追踪id代表一个触点，值-1表示一个未使用的slot。一个未出现的过的追踪id被认为是新的，而一个不在出现的追踪id考虑被移除。因为只传递了变化的信息，每个触点的全部信息需要保存在接收端。当接收到MT事件时，简单地更新当前slot的对应属性。当移动其中一个触摸点时，会看到如下的数据被传输。

   ABS_MT_SLOT 0
   ABS_MT_POSITION_X x[0]
   SYN_REPORT

可见只有变化的数据被传输，而不像协议A那样，每次传输需要发送所有的数据给上层。这就显著的减少了数据的传送量。

因此协议A和协议B的区别在于协议B可以支持由硬件设备来识别和追踪触摸点并提供ABS_MT_TRACKING_ID，这样可以减少数据传输量以及上层的运算量。

此外协议还定义了触摸事件的属性，一个触摸事件最少应该包括

事件类型	含义
ABS_MT_POSITION_X	接触面的形心的Ｘ坐标值
ABS_MT_POSITION_Y	接触面的形心的Ｙ坐标值
ABS_MT_TOUCH_MAJOR	触点主轴长度
ABS_MT_TOUCH_MINOR	触点的短轴长度
ABS_MT_PRESSURE	接触区域上的压力
ABS_MT_TOOL_TYPE	接触工具类型（比如手指和笔）
ABS_MT_TRACKING_ID	为接触点分配的ID

以上为多点触摸协议的基本内容，目前触控设备大多具备硬件识别并区分触摸点的能力，因此协议A基本上已经很少使用了。基于协议B的驱动代码应该是基于如下的传输步骤。

//设定slot
input_mt_slot(para = ABS_MT_TRACKING_ID); 
//设定slot的状态，slot 由input_mt_slot()函数指定
input_mt_report_slot_state(para = ABS_MT_TOOL_TYPE, para = STATUS)
//发送坐标信息 
input_report_abs(para = ABS_MT_POSITION_X) 
input_report_abs(para = ABS_MT_POSITION_Y)
//传输结束
input_sync(input_dev)

以下是ATMEL maXTouch IC 驱动的数据发送部分代码。

static void mxt_input_report(struct mxt_data *data, int single_id)
{
	struct mxt_finger *finger = data->finger;
	struct input_dev *input_dev = data->input_dev;
	int status = finger[single_id].status;
	int finger_num = 0;
	int id;

	for (id = 0; id < MXT_MAX_FINGER; id++) {
		if (!finger[id].status)
			continue;
		input_mt_slot(input_dev, id);
		input_mt_report_slot_state(input_dev, MT_TOOL_FINGER,
				finger[id].status != MXT_RELEASE);

		if (finger[id].status != MXT_RELEASE) {
			finger_num++;
			input_report_abs(input_dev, ABS_MT_TOUCH_MAJOR,
					finger[id].area);
			input_report_abs(input_dev, ABS_MT_POSITION_X,
					finger[id].x);
			input_report_abs(input_dev, ABS_MT_POSITION_Y,
					finger[id].y);
			input_report_abs(input_dev, ABS_MT_PRESSURE,
					finger[id].pressure);
		} else {
			finger[id].status = 0;
		}
	}

	input_report_key(input_dev, BTN_TOUCH, finger_num > 0);

	if (status != MXT_RELEASE) {
		input_report_abs(input_dev, ABS_X, finger[single_id].x);
		input_report_abs(input_dev, ABS_Y, finger[single_id].y);
		input_report_abs(input_dev,
				 ABS_PRESSURE, finger[single_id].pressure);
	}

	input_sync(input_dev);
}

以上代码很好的示范了我们这篇文章所讲的内容。

Linux 添加中断处理

David Dong — Thu, 09 Jan 2014 23:54:06 +0800

Linux 中断子系统的内部实现机制比较复杂，如果想讲清楚需要较长的篇幅才行，本文只从应用的角度介绍如何注册中断并添加中断处理函数到代码中，不对其内部的实现机制做太多阐述。

下面以我自己的代码为例，说明一下如何添加中断的处理。

1. 中断

中断是指在CPU正常运行期间，由于内外部事件或由程序预先安排的事件引起的CPU暂时停止正在运行的程序，转而为该内部或外部事件或预先安排的事件服务的程序中去，服务完毕后再返回去继续运行被暂时中断的程序。

2. 中断初始化

以下代码实现了注册一个中断处理到Linux系统中，省略了无关部分。

static int dev_probe(struct platform_device *pdev)
{
	struct device *dev = &pdev->dev;
	struct dev_ctl_data *dev_ctl = devm_kzalloc(dev, sizeof(*dev_ctl),
	GFP_KERNEL);
	int rc;

	...
	mutex_lock(&dev_ctl->lock);
	...
	
    //apply gpio for interrupt receive
	rc = dev_request_named_gpio(dev-ctl, "fp,gpio_irq", &dev-ctl->irq_gpio);
	if (rc) {
		pr_err("irq gpio request failed\n");
		goto exit;
	}
	// interrupt initialize
	atomic_set(&dev_ctl->wakeup_enabled, 0);
	dev_ctl->irqf = IRQF_TRIGGER_RISING | IRQF_ONESHOT;
	
	if (of_property_read_bool(dev->of_node, "tp,enable-wakeup")) {
		irqf |= IRQF_NO_SUSPEND;
		device_init_wakeup(dev, 1);
	}

	rc = devm_request_threaded_irq(dev, gpio_to_irq(dev-ctl->irq_gpio),
				NULL, dev_irq_handler, dev-ctl->irqf,
				dev_name(dev), dev-ctl);
	if (rc) {
		pr_err("could not request irq %d\n", gpio_to_irq(dev-ctl->irq_gpio));
		goto exit;
	}

	dev_dbg(dev, "requested irq %d\n", gpio_to_irq(dev-ctl->irq_gpio));

	/* Request that the interrupt should be wakeable */
	enable_irq_wake(gpio_to_irq(dev-ctl->irq_gpio));

	wakeup_source_init(&dev-ctl->ttw_wl, "fpc_ttw_wl");
	...
}

static int dev_request_named_gpio(struct dev-ctl_data *dev-ctl,
	const char *label, int *gpio)
{
	struct device *dev = dev-ctl->dev;
	struct device_node *np = dev->of_node;
	int rc = of_get_named_gpio(np, label, 0);
	if (rc < 0) {
		dev_err(dev, "failed to get '%s'\n", label);
		return rc;
	}
	*gpio = rc;
	rc = devm_gpio_request(dev, *gpio, label);
	if (rc) {
		dev_err(dev, "failed to request gpio %d\n", *gpio);
		return rc;
	}
	dev_dbg(dev, "%s %d\n", label, *gpio);
	return 0;
}

主要分为这么几个部分。

2.1 申请GPIO做为中断源

首先需要为中断申请一个GPIO，用来接收外部中断触发。代码中使用了dev_request_named_gpio()函数来申请这个GPIO。该函数为自定义函数，函数中调用了系统服务of_get_named_gpio()和devm_gpio_request()。其中参数 const char *label 为该GPIO的名字，需要和DTS中注册的名字一致，int *gpio 为返回的GPIO的索引号。 of_get_named_gpio(): 由名称索引到DTS中的GPIO，并返回GPIO号。
devm_gpio_request(): 在系统中分配GPIO给到指定的GPIO号。

2.2 设定中断属性

申请到GPIO后需要配置中断的FLAG,程序中该FLAG由dev_ctl->irqf指定。
常见的 FLAG 属性值有以下几个

类型	描述
IRQF_TRIGGER_RISING	上升沿触发中断
IRQF_TRIGGER_FALLING	下降沿触发中断
IRQF_TRIGGER_HIGH	高电平触发中断
IRQF_TRIGGER_LOW	低电平触发中断
IRQF_ONESHOT	单次触发，保证`thread_handler`函数执行完整，才会接受下一个中断信号
IRQF_NO_SUSPEND	在系统suspend的时候，不用disable这个中断
IRQF_SHARED	允许在多个设备中共享中断

以上代码中设定FLAG为 IRQF_TRIGGER_RISING 和 IRQF_ONESHOT, 如果指定可以唤醒系统的话（如代码，需要在sysfs节点中写入wakeup_enabled 为 1), 则该中断需要添加属性 IRQF_NO_SUSPEND。

2.3 注册中断，并绑定到中断服务程序

调用系统服务devm_request_threaded_irq()来实现该步骤。这个函数将GPIO注册到系统中断中并指定了中断服务程序dev_irq_handler()。若接收到该GPIO上的中断触发信号，系统会保存现场并跳转到dev_irq_handler()函数中执行相应的处理。

2.4 使能中断

代码中通过调用系统服务enable_irq_wake()来使能中断，enable_irq_wake()函数标记此中断可用于唤醒系统，与disable_irq_wake()配对，对于需要唤醒系统的中断，只需要在注册的时候调用此接口即可。

至此，中断在系统中的注册已经完成了，如果GPIO上有中断信号产生，那么系统会跳转到中断处理程序执行。我们来看一下中断处理程序。

3. 中断处理程序

static irqreturn_t dev_irq_handler(int irq, void *handle)
{
	struct dev-ctl_data *dev-ctl = handle;

	dev_dbg(dev-ctl->dev, "%s\n", __func__);

	mutex_lock(&dev-ctl->lock);
	if (atomic_read(&dev-ctl->wakeup_enabled)) {
		dev-ctl->nbr_irqs_received++;
       //wakeup system
		__pm_wakeup_event(&dev-ctl->ttw_wl, TTW_HOLD_TIME);
	}
	mutex_unlock(&dev-ctl->lock);
    //wakeup user proc that blocked by waiting sys node.
	sysfs_notify(&dev-ctl->dev->kobj, NULL, dev_attr_irq.attr.name);

	return IRQ_HANDLED;
}

同单片机的中断处理程序不同，Linux的中断处理程序会有一个int类型的返回值。中断服务程序中一般放置需要即时处理的事情。这里的处理非常简单，调用了__pm_wakeup_event ( &dev-ctl->ttw_wl, TTW_HOLD_TIME )来唤醒系统，唤醒源为在初始化函数中wakeup_source_init(&dev-ctl->ttw_wl, "fpc_ttw_wl")定义的ttw_wl。并且发送一个系统的通知sysfs_notify(),用来唤醒在读写属性文件(sysfs节点)时因调用select()或poll()而阻塞的用户进程。

浅谈触摸屏充电器噪声要求及测量方法

David Dong — Sun, 15 Dec 2013 23:01:16 +0800

触摸屏在高噪声环境下的性能是当前移动电子设备特别是智能手机设计人员面临的巨大挑战之一。而在这些噪声当中，充电器噪声是被广泛涉及和对触摸屏影响最大的噪声源。这种噪声通过USB充电器物理耦合至传感器进而被TOUCH IC所采集。它可以造成触摸的精度或线性度下降、产生误触，严重的会造成触摸屏变得无响应。本文结合我在工作中的一些经验来介绍一下触摸屏设计中的对噪声相关的要求，测量方法以及一些降噪的方法。

1. 概述

当前业界广泛应用的标准是TOUCH IC 能够在40V Vpp 充电器噪声的环境下检测到微库仑级别的电荷变化。
如图所示，一个大噪声的充电器可以将共模噪声注入到触摸屏系统中。

2. 触摸屏电源输入端的要求

通常来说充电器噪声的幅度大小和频谱分布对于触摸屏IC的工作情况有较大的影响。因此对于任何触控IC 来说，对噪声的幅度和频谱分布都会有要求。

首先对于充电器共模噪声幅度的要求是其要低于一定的阈值（根据IC的噪声抑制能力不同，阈值会有不同）; 同时不同频带的噪声的阈值有较大区别。另外对于共模噪声的频谱的要求是在和电容屏触控IC 的使用频带重合的频带内, 具有一定宽度的干净频带。

以上两个方面的条件必须同时得到满足, 才达到电源引脚的输入共模噪声要求. 如果某充电器只满足第一方面的要求不满足第二方面的要求, 即噪声幅度峰峰值没有超标, 但在IC 工作可选有效频段范围内不存在干净区域, 则该充电器不达标; 如果某充电器只满足第二方面的要求但不满足第一方面的要求, 即在IC 可选有效频段范围内存在符合条件的干净区域, 但噪声的幅度超标, 这样的充电器也不达标。在电网环境较差的情况下, 充电器的输出噪声可能会明显大过该充电器在普通市电网环境下的噪声。

为了规范充电器噪声的标准，许多 OEM 厂商联合起来创建了 EN 规范，这些规范对充电器在任何频率下发出的最大噪声水平进行了限定。请参见EN 62684-2010 和 EN 301489-34v1.1.1。

规范中规定在1k Hz到10k Hz 频谱范围内，充电器噪声的峰峰值不应该大于 1V Vpp。在高于10k Hz 的频谱范围，充电器输出的噪声的峰峰值应该随着频率的升高按指数级降低。需要指出的是该规范较为严格，目前市场上多数的充电器都没有严格按照该规范的要求设计，从而造成输出的噪声远远大于规范的要求。目前主流的触控IC厂家内部要求IC 在1k Hz – 400k Hz的工作频段下能够抗住至少40V Vpp的充电器噪声。例如CYPRESS 可以满足95v Vpp 以上的抗噪要求，ATMEL的Maxcharger 技术可以在某些频段上实现最大240v Vpp噪声下的良好工作。

3. 充电器噪声的测量方法

充电器的噪声属于共模噪声，噪声源会造成系统GND的浮动，从而产生一个 Vpp-noise大小的电压偏差。

基于以上模型，可以按照如下步骤测量充电器耦合进系统的噪声。

把充电器接上设备, 让充电器处于负载工作状态。
示波器调到FFT分析窗口。
示波器/频谱分析仪的探头的正极与待测设备的直流GND 相连; 探头的负极不要和待测设备相连, 可以和示波器/频谱分析仪的大地相连。
确保示波器的GND和大地相连。

4. 对触摸屏设计的一些要求

除了触控IC本身的抗噪声能力外，触碰传感器的设计对于系统抗噪声的性能也息息相关。当前的投射电容传感器设计由TX （发射极）和RX（检测极）构成。通过检测TX和RX之间形成的耦合电容上的电荷变化来感知物体的触碰。检测电荷的变化只有微库仑级别，其中RX线由于是信号检测线所以对于耦合进来的噪声尤其敏感。

通常要考虑RX ITO 的面积和线阻抗对抗噪声的影响。

理想的设计是RX 线要有较小的线阻抗来保证较小的充电时间，从而避免在充电期间导入过多的噪声信号。同时也要保证RX线要有小的面积，来减少噪声的引入。我们知道，ITO线的阻抗和其宽度相关，相同的ITO材料越宽的走线其线阻抗也越低。因此在设计时要平衡好阻抗和图形的关系。好在现在各家触控IC厂都有自己的设计规范在保证达到最好的抗噪性能。

下面是两个触控IC厂家对于触摸屏设计的一些规范。

厂家1 对于ITO阻抗的要求

不建议使用菱形图案的屏体图案设计, 因为该设计较宽的RX面积一般难以达到抗噪声的性能要求。 *注RX电阻包括了ITO 电阻+走线 (如金属走线) 电阻. 对于尺寸较大的屏(如7’’及7’’以上的TP), RX 单边走线难以达到<10K Ohm 的要求, 强烈建议RX 通道采用双边走线的方式。

厂家2 对于 ITO 图案的改进对传统的diamond图案做了改进，减少了RX的宽度。

5. TOUCH IC 的降噪方法

现在主流的触控IC供应商都研发了各家的降噪算法，像ATMEL Maxcharger 和CYPRESS Charger Armor 等。

通常这些方法都采用提高前端模拟电路的信躁比和后端各种滤波算法的应用。

前端模拟电路信躁比信噪比是触摸屏控制器的性能指标，现在已经作为行业标准被大家接受。在电容式触摸屏中的信躁比定义为触摸的信号和叠加在信号之上的噪声信号的峰峰值之间的比值。信号就是加上测量到的手指电容后的实际电容的变化量。手指电容取决于传感器覆盖物厚度、手指大小，DUT到地的寄生电容，以及传感器模式。噪声成分依赖于内部控制器噪声和外部噪声源。由于触摸控制 IC 模拟端得到的原始信噪比与触摸屏的驱动电压成正比，因此可以期望采用高驱动电压 (TX)的方式来提高系统的信噪比。一般的触摸屏控制器是以设备电压2.7v来驱动触摸屏，为了提高驱动电压ATMEL设计了内部电压泵可以采用DOUBLE驱动模式，从而将驱动电压提高到7v左右。而赛普拉斯的 Gen4同样采用 2.7 V 的模拟电源，但实际上以 10 V 的电压驱动触摸屏。Gen4 控制器内部集成了电荷泵和 10v 晶体管，这使得它们能够实现 3.7 倍于其他任何芯片的原始信噪比。需要注意的是高的驱动电压同样会导致系统功耗的增加，在某些场合下会对其他电路如LCD电路等产生干扰。这些在设计时都需要仔细考虑。值得一提的是ATMEL独有的DUAL-X 双线驱动扫描模式，能够在不提高驱动电压的模式下将SNR提高到原来的1.6倍。
后端处理一旦获取信号后，就可以采用中值滤波或其他更先进的非线性滤波等典型技术来进一步提高信噪比，但这将以牺牲刷新率为代价。ATMEL的maXTouch IC 就采用了中值滤波的算法，在系统处于大噪声耦合的环境下用户可以选择使用该滤波器，代价是刷新率会降到正常时的1/3。但是在某些场合下如果设备需处理的带内噪声很大时，即使采用滤波算法也很难得到干净的信号值，这时一些特殊的方法会派上用场。例如ATMEL maXTouch的自适应跳频技术是解决触摸屏中的充电器噪声问题的另一项关键技术。他能够结合背景噪声的测量扫描动态的调整其扫描驱动 (Tx) 频率并调整其ADC采样的个数来抑制噪声。

6. 总结

随着智能手机的广泛普及，电容触摸屏得到了越来越多的应用，同时对其抗噪的能力也提出了更大的挑战。当前的趋势是要求TOUCH IC具有更快的对噪声作出反应的能力，在不增加过多成本的前提下能实现更小的功率消耗同时不会过多的降低系统的报点率。尽管在整个嵌入式系统中，电容触摸屏实现了一些最先进的功能，包括FIR 滤波器，高级非线性滤波方法、内置抗噪声硬件、跳频功能或其他方法等。但是但抗噪声能力的提升仍然具有很大的提升空间。各家TOUCH IC厂家也在积极的研发创新，新的降噪算法会不断出现。

Linux fork() 函数

David Dong — Sun, 22 Sep 2013 20:12:01 +0800

本文主要总结了Linux fork()函数的使用。

在Linux中每个进程拥有独立的地址空间,地址空间包括代码段、数据段和堆栈段。

代码段：存储程序的代码
数据段：存储程序的全局变量和动态分配的内存
堆栈段：存储函数运行时的局部变量

进程之间的地址空间是隔离的。

fork()函数的作用是新建立一个进程。这个进程是从父进程COPY过来，因此和父进程几乎完全相同。 fork()函数的原型如下,需要注意的是函数的返回值。

pid_t fork(void);

函数返回值:

0:子进程返回0
-1:出错返回
其它：父进程返回子进程ID

注意，fork被调用后，是要返回两次，也就是有两个返回值， 分别代表了父进程和子进程，这点和一般的函数调用不同。

看代码。

#include <unistd.h>  
#include <stdio.h>   
int main ()   
{   
    pid_t fpid; 
    int mCnt=0; 
    //调用fork()
    fpid=fork();
 
    //fpid 返回0 代表error
    if (fpid < 0){   
        printf("error in fork!"); 
    }  
    // fpid 等于0代表执行的子进程
    else if (fpid == 0) {  
        printf("子进程：id is %4d, pid is %4d, ppid is %4d,/n",fpid,getpid(),getppid());   
        mCnt++;  
    }  
    // 代表父进程执行
    else {  
        printf("父进程：id is %4d, pid is %4d, ppid is %4d,/n",fpid,getpid(),getppid());   
        mCnt++;  
    }  
    printf("执行了: %d/n 次",mCnt);  
    return 0;     
}

执行结果如下。

子进程：id is    0, pid is 2621, ppid is 1158,执行了: 1 次
父进程：id is 2621, pid is 2620, ppid is 2019,执行了: 1 次

我们可以看到相同的代码段被执行了两次。这是因为调用fork()函数后，就变成两个进程在执行了，这两个进程的执行逻辑完全相同，在子进程中，fork函数返回0，在父进程中，fork返回新创建子进程的进程ID。我们可以通过fork返回的值来判断当前进程是子进程还是父进程。如果把代码稍作修改。

    else if (fpid != 0) {  
        printf("父进程：id is %4d, pid is %4d, ppid is %4d,/n",fpid,getpid(),getppid());   
        mCnt++;  
    }  
    else {  
        printf("子进程：id is %4d, pid is %4d, ppid is %4d,/n",fpid,getpid(),getppid());   
        mCnt++;  
    }

结果如下，可以看出到是子进程先执行，父进程后执行，与程序的执行顺序无关。

子进程：id is    0, pid is 2621, ppid is 1158,执行了: 1 次
父进程：id is 2621, pid is 2620, ppid is 2019,执行了: 1 次

关于fork()函数的执行原理，用到了“写时复制”的技术，参考以下的博文，讲的比较清楚。
https://www.cnblogs.com/zhangchaoyang/articles/2317420.html

fork（）会产生一个和父进程完全相同的子进程，但子进程在此后多会exec系统调用，出于效率考虑， linux中引入了“写时复制“技术，也就是只有进程空间的各段的内容要发生变化时，才会将父进程的内容复制一份给子进程。在fork之后exec之前两个进程用的是相同的物理空间（内存区），子进程的代码段、数据段、堆栈都是指向父进程的物理空间，也就是说，两者的虚拟空间不同，但其对应的物理空间是同一个。当父子进程中有更改相应段的行为发生时，再为子进程相应的段分配物理空间，如果不是因为exec，内核会给子进程的数据段、堆栈段分配相应的物理空间（至此两者有各自的进程空间，互不影响），而代码段继续共享父进程的物理空间（两者的代码完全相同）。而如果是因为exec，由于两者执行的代码不同，子进程的代码段也会分配单独的物理空间。 fork之后内核会通过将子进程放在队列的前面，以让子进程先执行，以免父进程执行导致写时复制，而后子进程执行exec系统调用，因无意义的复制而造成效率的下降。 fork时子进程获得父进程数据空间、堆和栈的复制，所以变量的地址（当然是虚拟地址）也是一样的。

参考：
https://www.cnblogs.com/zhangchaoyang/articles/2317420.html https://blog.csdn.net/zhangxiao93/article/details/72811700

poll() 函数使用

David Dong — Mon, 02 Sep 2013 20:12:01 +0800

这篇文章对Linux的poll()函数的使用做一个总结。

poll() 函数

Linux 中 poll() 函数用来监听并等待多个文件描述符的属性变化。
原型

int poll(struct pollfd *fds, nfds_t nfds, int timeout);

其中fds：指向一个结构体数组的第0个元素的指针，每个数组元素都是一个struct pollfd结构，用于指定测试某个给定的fd的条件。

struct pollfd{
	int fd;			/*文件描述符*/
	short events;	/*等待的事件*/
	short revents;	/*实际发生的事件*/
};

fds 结构体参数说明

fd：每一个 pollfd 结构体指定了一个被监视的文件描述符，可以传递多个结构体，指示 poll() 监视多个文件描述符。
events：指定监测fd的事件（输入、输出、错误），每一个事件有多个取值，如下
revents：revents 域是文件描述符的操作结果事件，内核在调用返回时设置这个域。events 域中请求的任何事件都可能在 revents 域中返回。注意：每个结构体的 events 域是由用户来设置，告诉内核我们关注的是什么，而 revents 域是返回时内核设置的，以说明对该描述符发生了什么事件。
nfds：用来指定第一个参数数组元素个数。
timeout：指定等待的毫秒数，无论 I/O 是否准备好，poll() 都会返回。

返回值：

成功时，poll() 返回结构体中 revents 域不为 0 的文件描述符个数；如果在超时前没有任何事件发生，poll()返回 0；
失败时，poll() 返回 -1；
并设置 errno 为下列值之一：

EBADF：一个或多个结构体中指定的文件描述符无效。
EFAULT：fds 指针指向的地址超出进程的地址空间。
EINTR：请求的事件之前产生一个信号，调用可以重新发起。
EINVAL：nfds 参数超出 PLIMIT_NOFILE 值。
ENOMEM：可用内存不足，无法完成请求。

MISRA C 规则

David Dong — Thu, 13 Jun 2013 22:41:23 +0800

最近工作中有接触到MISRA C的规范，为了方便以后查找，这里把规则都记录下来。

MISRA C编程规范（MISRA C Coding Standard），是针对C 语言编程的软件开发标准。它的目标促进C代码在嵌入式系统中的安全性、可移植性和可靠性。

以下内容为转贴。

Rule 1. 严格遵循ANSI C89标准，不允许任何扩展。

Rule 3. 如果要嵌入汇编语言，则必须将所有汇编语句包装在C函数里，而且这些函数中只有汇编语句，没有常规C语句。

Rule 7. 不得使用三元操作符 （? : )

Rule 10. 不得残留被注释掉的废代码。

Rule 11. 所有标识符不超过31字符。

Rule 12. 不同名空间中的变量名不得相同。例如：

typedef struct MyStruct {... } MyStruct;  (违规）

struct Person {
    char* name;
    ...
};
char name[32];  （违规）

Rule 13. 不得使用 char, int, float, double, long 等基本类型，应该用自己定义的类型显示表示类型的大小，如 CHAR8, UCHAR8, INT16, INT32, FLOAT32, LONG64, ULONG64 等。

Rule 14. 不得使用类型 char，必须显示声明为 unsigned char 或者 signed char。

Rule 18. 所有数字常数应当加上合适的后缀表示类型，例如 51L, 42U, 34.12F 等。

Rule 19. 禁止使用八进制数。（因为 086U 这样的常数很容易引起误解）。

Rule 21. 不得定义与外部作用域中某个标识符同名的对象，以避免遮盖外部作用域中的标识符。

Rule 23. 具有文件作用域的对象尽量声名为 static 的。

Rule 24. 在同一个编译单元中，同一个标识符不应该同事具有内部链接和外部链接的声名。

这里我略作说明：我们通常将一些放在头文件里的变量声名为“外部链接”的，如：

  extern UINT32 g_count;  // 俗话叫变量声明（对应于变量定义，不分配实际空间）

对于“使用”这个变量的.c文件来说，这很好，因为 g_count 始终保持外部链接性质。可是对于定义 g_count（实际分配空间）的 .c 文件来说，如果包含了上述的头文件，则在这个编译单元里就发生了内部链接和外部链接的冲突。

解决办法是，定义 g_count 的文件尽量不要包含声名 g_count 的头文件。个人感觉这不是任何时候都做得到的，尤其是在对付遗留代码的时候。

Rule 25. 具有外部链接性质的标识符应该只声明一次。

Rule 27. 外部对象不得在多个文件中声名。

Rule 28. 禁止使用 register 关键字。

Rule 29. 自动对象（栈对象）使用前必须赋初值。

Rule 33. 操作符 && 和 || 的右侧表达式不得具有副作用（side-effect）。也就是说，象

  if (x == 20 && ++y == 19)

这样的表达式被禁止。

Rule 35. 在返回布尔值的表达式中不得出现赋值操作。也就是说，我们常用的

if (!(fp = fopen("fname", "r"))) { /* error */ }

被禁止。

Rule 37. 不得对有符号数施加位操作，例如 1 << 4 将被禁止，必须写 1UL << 4;

Rule 39. 不得对有符号表达式施加一元 - 操作符。

Rule 40. 不得对有副作用的表达式施加 sizeof 操作符。

Rule 42. 除了循环控制语句，不得使用逗号表达式。

Rule 44. 禁止冗余的显式转型。

比如:

double pi = (double) 3.1416F;

Rule 45. 禁止从任意类型到指针的强制转型，禁止从指针到任意类型的强制转型。

例如：

void* p = (void*)0xFFFF8888UL;

Rule 49. 显示测试值是否为零。

Rule 50. 不得显式判断浮点数的相等性和不等性。

Rule 52. 不得遗留“永远不会用到”的代码。

Rule 53. 所有非空语句必须具有副作用。

Rule 55. 除了 switch 语句，不得使用标号 (label)。

Rule 56. 不得使用 goto.

Rule 57. 不得使用 continue。

Rule 58. 除了 switch 语句，不得使用 break.

Rule 59. if, else if, else, while, do..while, for 语句块必须使用 {} 括起。

Rule 60. 任何 if..else if 语句，最后必须有一个收尾的 else。

例如：

if (ans == 'Y') {
   ...
}
else if (ans == 'N') {
   ...
}
else {
   ;
}

Rule 67. 循环计数器的值不得在循环体内修改。

Rule 70. 禁止任何直接和间接的递归函数调用。

Rule 82. 每个函数只能有一个推出点。

Rule 86. 如果一个函数可能返回错误信息，则调用后必须加以测试。

Rule 92. 不应该使用 #undef。

Rule 95. 不得将宏作为参数传给宏函数

Rule 98. 在一个宏定义中，# 或 ## 符号只能出现一次。

Rule 101. 禁止指针运算（代之以数组下标运算）。

Rule 102. 禁止超过两级的指针。

Rule 104. 禁止使用指向函数的非常量指针。

Rule 106. 不得将栈对象的地址传给外部作用域的对象。

后面的规则针对实时嵌入式系统，对其他类型的开发未必适用，如：

Rule 118. 禁止使用动态堆分配（也就是不得使用 malloc, calloc 和 realloc）。

Rule 119. 禁止使用 errno。

Rule 120. 禁止使用 offsetof。

Rule 121. 禁止使用 <locale.h>。

Rule 122. 禁止使用 setjmp, longjmp。

Rule 123. 禁止使用 <signal.h>。

Rule 124. 禁止使用 <stdio.h>（不能用 printf, scanf了！）。

Rule 125. 禁止使用 atoi, atof, atol。（这个我很赞成，建议使用 strtol, strtod 等函数）。

Rule 126. 禁止使用 abort, exit, getenv。

Rule 127. 禁止使用 <time.h>。

Java 调用 COM 组件的方法

David Dong — Tue, 07 May 2013 19:11:17 +0800

最近项目中需要用 JavaSE 来调用 Windows 的下的程序代码。如 Java 编写的 GUI 需要操作 Windows 平台下自己开发的代码。这里总结一下相关的技术内容。

因为考虑到要跨语言的程序调用，就要用到 COM 组件。COM 组件是为了实现代码的跨平台/语言复用，提供一种独立于平台与编程语言之外的，共享二进制代码的方法。其实就是一种规范化的接口，基于DLL(动态链接库)的形式存在。因为是以二进制的形式传播的，因此在统一的规范接口下可以在不同的编程语言之间调用。微软为其 Windows 的一系列服务都提供了 COM 的接口，供第三方编程调用。当然，用户也可以定义编写自己的 COM 组件，实现代码的跨平台/语言的复用。基于 COM 的技术细节很多，这里不过多阐述。本文的目的是主要总结一下 Java 对于 COM 的调用方法及注意事项。

目前有一些 Java 库可以支持 Java 来调用 COM 的方案，包括一些第三方的商业的库和开源的库。因为Java 编程，基于开源的精神，当然使用开源的库了。我选择用 Jacob 库来实现。

以下是 Jacob 的介绍，来源于官方文档。

JACOB is a JAVA-COM Bridge that allows you to call COM Automation components from Java.
It uses JNI to make native calls into the COM and Win32 libraries.
The JACOB project started in 1999 and is being actively used by thousands of developers worldwide.
As an open-source project, it has benefited from the combined experience of these users, many of whom have made modifications to the code and submitted them back for inclusion in the project.

Jacob 通过 Jni 来实现对 COM 组件和 WIN32 库的调用。Jacob 提供了一系列的编程接口，编程者编程时只需要面对 Jacob 的接口来编程即可，无需关注底层的实现细节。

Jacob 的下载地址官方下载

下载下来的 package 里包含一个Jacob.jar的jar包和jacob-1.15-M4-x64.dll(64 bits)以及jacob-1.15-M4-x86.dll(32 bits)的两个 DLL 文件。分别定义了 Java 的接口和 32 位以及 64 位的 native 接口。

Jacob.jar
jacob-1.15-M4-x64.dll
jacob-1.15-M4-x86.dll

使用 Jacob 的步骤如下。

拷贝 jacob-1.15-M4-x64.dll (64 bits) and jacob-1.15-M4-x86.dll (32 bits) 到 windows/system32/ 目录下。或者拷贝到Java 的类路径下 Program Files/Java/jdk1.6.0_14/jre/bin/（推荐使用）
把 jacob.jar 导入项目的classpath Program Files/Java/jdk1.6.0_14/jre/lib/ext/
(注：导入外部 jar 都用类似方法，例如调用串口支持包 Comm.jar win32com.dll)
把要使用的 Dll 进行注册，
C:\Windows\Microsoft.NET\Framework\v4.0.30319\regsvcs.exe [需要调用的dll文件路径]
C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\RegAsm.exe [需要调用的dll文件路径]
注意这里要先regasm xxx.dll，再regasm xxx.dll /tlb:xxx.tlb /codebase，要注册两个。

具体可以参考如下的网页。
http://msdn.microsoft.com/zh-cn/library/h627s4zy.aspx
http://www.haogongju.net/art/130918

调用的代码如下，我自己实现的操作类，有兴趣的可以自己下载使用。

ServiceComInterface.java (定义接口)

/*

* To change this template, choose Tools | Templates
* and open the template in the editor.
  */
  package org.dong.service.com;

import com.jacob.activeX.ActiveXComponent;

/**
 *

* @author Dong Gang
  */
  public interface ServiceComInterface {
   public ActiveXComponent getAXC(String dllname);
   public String executeMethod(String methodname, String param);

}

ServiceCom.java (具体的实现类)

/*

* To change this template, choose Tools | Templates
* and open the template in the editor.
  */
  package org.dong.service.com;

import com.jacob.activeX.ActiveXComponent;
import com.jacob.com.ComException;
import com.jacob.com.ComThread;
import com.jacob.com.Dispatch;
import com.jacob.com.Variant;
import javax.swing.JOptionPane;

/**
 *

* @author Dong Gang
  */
  public class ServiceCom implements ServiceComInterface {
  
   private static ServiceCom serviceComInstance = null;
   //private static ActiveXComponent axc = null;
   private ActiveXComponent axc = null;
   private ServiceCom() {
   }
  
   private ServiceCom(String dllname) {
  
       getAXC(dllname);
  
   }
  
   public static ServiceCom getInstance() {
  
       if (serviceComInstance == null) {
           serviceComInstance = new ServiceCom();
       }
       return serviceComInstance;
  
   }
  
   public static ServiceCom getInstance(String dllname) {
  
       //if (serviceComInstance == null) {
       serviceComInstance = new ServiceCom(dllname);
       //}
       return serviceComInstance;
  
   }
  
   public void clear() {
  
       serviceComInstance = null;
  
   }
  
   @Override
   public ActiveXComponent getAXC(String dllname) {
  
       try {
           axc = new ActiveXComponent(dllname);
       } catch (Exception e) {
           JOptionPane.showMessageDialog(null, "load com error!",
           "Error!", JOptionPane.ERROR_MESSAGE);
           ComThread.Release();
       } finally {
       }
       return axc;
  
   }
  
   @Override
   public String executeMethod(String methodname, String param) {
  
       try {
           ActiveXComponent axc1 = axc;
           String msg = Dispatch.call(axc1, methodname, param).toString();
           // ComThread.Release();
           return msg;
       } catch (ComException e) {
           e.printStackTrace();
       }
       return null;
  
   }
  
   // @SuppressWarnings("CallToThreadDumpStack")
   public String executeMethod(String methodname) {
  
       try {
           ActiveXComponent axc1 = axc;
      
           //Variant v = Dispatch.call(axc1, methodname);
           //String msg = v.toString();
      
           String msg = Dispatch.call(axc1, methodname).toString();
           // ComThread.Release();
           return msg;
       } catch (ComException e) {
           JOptionPane.showMessageDialog(null, "com execute error!",
           "Error", JOptionPane.ERROR_MESSAGE);
           e.printStackTrace();
       }
       return null;
  
   }
  
   public String executeMethod(String methodname, Object[] params) {
  
       try {
      
           ActiveXComponent axc1 = axc;
           String abc = Dispatch.callN(axc1, methodname, params).toString();
           // ComThread.Release();
           return abc;
       } catch (ComException e) {
           e.printStackTrace();
       }
       return null;
  
   }
  
   public String executeMethod(String methodname, String param1, String param2) {
  
       try {
           ActiveXComponent axc1 = axc;
           String msg = Dispatch.call(axc1, methodname, param1, param2).toString();
           // ComThread.Release();
           return msg;
       } catch (ComException e) {
           e.printStackTrace();
       }
       return null;
  
   }
  }

Java JDOM 库的使用

David Dong — Fri, 01 Mar 2013 09:01:11 +0800

最近的开发工作中使用到了JDOM工具包来编辑XML类型的文档。JDOM是一种使用 XML 的Java 工具包。基于 Java 平台并通过 Java 代码来访问、操作并输出 XML 数据。

以下的介绍转载自百度百科, JDOM工具包的下载地址

JDOM是一个开源项目，它基于树型结构，利用纯JAVA的技术对XML文档实现解析、生成、序列化以及多种操作。JDOM 直接为JAVA编程服务。它利用更为强有力的JAVA语言的诸多特性（方法重载、集合概念以及映射），把SAX和DOM的功能有效地结合起来。
Jdom是用Java语言读、写、操作XML的新API函数。Jason Hunter 和 Brett McLaughlin 公开发布了它的1.0版本。在直觉、简单和高效的前提下，这些API函数被最大限度的优化。

XML 类型文档是一个树状结构，因此在创建或者访问是要首先定位到根节点。然后依照分支去搜寻要定位的节点。

以下是我实现的用JDOM 操作XML文档的示列代码。

ServiceXmlInterface.java

/*
 * To change this template, choose Tools | Templates
 * and open the template in the editor.
 */
package org.dong.service.xml;

/**
 *
 * @author Dong Gang
 */
public interface ServiceXmlInterface {
    public void generateXml(String path,String content);
}

ServiceXml.java 封装XML文档的操作，实现了ServiceXmlInterface 接口。

package org.dong.service.xml;
import org.jdom.Attribute;
import org.jdom.Comment;
import org.jdom.Document;
import org.jdom.Element;
import org.jdom.JDOMException;
import org.jdom.input.SAXBuilder;
import org.jdom.output.Format;
import org.jdom.output.XMLOutputter;
import org.openide.util.Exceptions;

public class ServiceXml implements ServiceXmlInterface {

    private static ServiceXml serviceXmlInstance = null;
    private Document doc;
    private Element rootElement;

    private ServiceXml() {
    }
	//单例模式，保证全局唯一句柄
    public static ServiceXml getInstance() {
        if (serviceXmlInstance == null) {
            serviceXmlInstance = new ServiceXml();
        }
        return serviceXmlInstance;
    }

    //读取XML文档
    public void loadConfigurationFile(String path) {

        SAXBuilder builder = new SAXBuilder(false);
        try {
            try {
                InputStreamReader isr = new InputStreamReader(new FileInputStream(path), "gbk");

                try {
                    doc = builder.build(isr);
 					//得到根节点
                    rootElement = doc.getRootElement();
					//得到子节点的列表
                    List<Element> rootList = rootElement.getChildren();
					
                    //以下遍历子节点的内容
                    if (!rootList.isEmpty()) {
                        Element fileList = rootElement.getChild("files-list");
                        List<Element> fileElements = fileList.getChildren();
                        List<String> fileNames = new ArrayList<String>();
                        for (int i = 0; i < fileElements.size(); i++) {
                            Element fileElement = fileElements.get(i);
                            String fileName = fileElement.getAttributeValue("name");
                            fileNames.add(fileName);
                        }
                        ConstantFactory.globalConfStore.setFiles(fileNames);

                        Element uiStyleElement = rootElement.getChild("ui-style");
                        String uiStyle = uiStyleElement.getAttributeValue("name");
                        ConstantFactory.globalConfStore.setUiStyle(uiStyle);

                        Element ruleChk = rootElement.getChild("check-warning");
                        String chkWarningInfo = ruleChk.getAttributeValue("value");
                          
                        ConstantFactory.globalConfStore.setExeWarningChk(chkWarningInfo);

                        Element formatElement = rootElement.getChild("format");
                        String format = formatElement.getAttributeValue("value");
                        ConstantFactory.globalConfStore.setFormat(format);

                        Element recordNumberElement = rootElement.getChild("record-number");
                        String recordNumber = recordNumberElement.getAttributeValue("value");
                        ConstantFactory.globalConfStore.setRecordNumber(recordNumber);

                    } else {
                    }
                } catch (IOException ex) {
                    Logger.getLogger(XMLParser.class.getName()).log(Level.SEVERE, null, ex);
                } catch (JDOMException e) {
                    e.printStackTrace();
                }

            } catch (UnsupportedEncodingException ex) {
                Exceptions.printStackTrace(ex);
            }
        } catch (FileNotFoundException ex) {
            Exceptions.printStackTrace(ex);
        }
    }

}
//生成XML Document 实例，返回值为一个XML Document 类型的对象
public Document generateConfigurationFile(GlobalConfigurationStore gcs) {

        Document doc;
        //生成根节点
        Element rootElement = new Element("global-configuration");
        doc = new Document(rootElement);
        //根节点内容
        rootElement.addContent(new Comment("Global Configuration infomation."));
        Element fileList = new Element("files-list");
        fileList.setAttribute(new Attribute("count", String.valueOf(gcs.getFiles().size())));
        rootElement.addContent(fileList);
        List<String> fileNames = gcs.getFiles();
        //生成子节点
        for (int i = 0; i < fileNames.size(); i++) {
            Element fileElement = new Element("file");
            fileElement.setAttribute(new Attribute("name", fileNames.get(i)));
            fileList.addContent(fileElement);
        }

        Element uiStyleElement = new Element("ui-style");
        String uiStyle = gcs.getUiStyle();
        uiStyleElement.setAttribute(new Attribute("name", uiStyle));
        rootElement.addContent(uiStyleElement);

        Element ruleChk = new Element("check-warning");
        String checkWarning = gcs.getExeWarningChk();
        ruleChk.setAttribute(new Attribute("value", checkWarning));
        rootElement.addContent(ruleChk);

        Element formatElement = new Element("format");
        String format = gcs.getFormat();
        formatElement.setAttribute(new Attribute("value", format));
        rootElement.addContent(formatElement);

        Element recordNumberElement = new Element("record-number");
        String recordNumber = gcs.getRecordNumber();
        recordNumberElement.setAttribute(new Attribute("value", recordNumber));
        rootElement.addContent(recordNumberElement);

        return doc;
    }

//由Document 对象生成一个XML文档
 public void generateXml(String path, Document document) {
        try {
            File testFile = new File(path);
            FileWriter hWriter = new FileWriter(testFile);
            Format format = Format.getPrettyFormat();
            format.setIndent("    ");
            format.setEncoding("gbk");
            //格式化
            XMLOutputter out = new XMLOutputter(format);

            if (!testFile.exists()) {
                testFile.createNewFile();
            } else {
            }
            //输出文件
            out.output(document, hWriter);
            hWriter.close();
        } catch (Exception e) {
            e.getMessage();
        }
    }

如以上代码，使用JDOM解析一个XML文档的步骤如下。

实例化一个合适的解析器对象,本例中我们使用SAXBuilder

SAXBuilder builder = new SAXBuilder(false);

以包含XML数据的文件为参数，构建一个文档对象myDocument

InputStreamReader isr = new InputStreamReader(new FileInputStream(path), "gbk");
Document doc = builder.build(isr);

找到根元素

Element rootElement = doc.getRootElement();

一旦你获取了根元素，你就可以很方便地对它下面的子元素进行操作了，

List<Element> rootList = rootElement.getChildren();

下面对Element对象的一些常用方法作一下简单说明：

方法	说明
getChild	返回指定名字的子节点,如果同一级有多个同名子节点，则只返回第一个,如果没有返回null值。
getChildren	返回指定名字的子节点List集合。这样你就可以遍历所有的同一级同名子节点。
getAttributeValue	返回指定属性名字的值。如果没有该属性则返回null,有该属性但是值为空，则返回空字符串。
getChildText	返回指定子节点的内容文本值。
getText	返回该元素的内容文本值。
setText	修改该元素的内容文本值。
remove	删除第index+1个子元素
removeAll	删除Children的子元素
removeChildren	删除名字为childname的子元素
add	添加Element

Java 使用 JXL 库编辑 Excel 文档

David Dong — Mon, 17 Dec 2012 17:25:19 +0800

因为开发中用到了 JXL 库来处理 Windows 的 Excel 文档，这里做一个总结记录，以方便未来查找。

JXL.jar 是一个 Java 类包，封装了对 Excel 文档的操作。提供了一系列接口，方便编程者读写编辑 Excel 类型的文档。

JXL 的官方下载地址官方地址

将下载后的文件解包，得到 JXL.JAR，放入 classpath。如果包不能正常的加载，可以把你下载好的 JXL.JAR 包放入本机 jre 的安装目录下。

直接讲用法不如看代码，以下是一些使用 JXL 库的简单操作。

导入jxl的类库

import jxl.Cell;
import jxl.Sheet;
import jxl.Workbook;
import jxl.write.Label;
import jxl.write.WritableSheet;
import jxl.write.WritableWorkbook;

打开读取Excel表格的方法

try{

  //声明一个workbook的对象，workbook代表选取Excel文件的工作薄
  Workbook book = Workbook.getWorkbook(fileChooser.getSelectedFile());
  //Sheet对象表示当前工作中的一页，getSheet 参数代表页码的索引
  Sheet sheet = book.getSheet(0);
  //Cell 对象代表一个单元格。getCell 参数为该单元格的行和列索引
  Cell cell = sheet.getCell(0, 1);
  //分别读取单元格（0，1）（1，1）（2，1）的内容
  String strIndex = cell.getContents();
  cell = sheet.getCell(1, 1);
  String strTime = cell.getContents();
  cell = sheet.getCell(2, 1);
  String strRefSpeed = cell.getContents();

  ...
}
catch(Exception e){
  e.printStackTrace();
}

创建一个Excel workbook的方法

try {
  // 建立文件
  WritableWorkbook book = Workbook.createWorkbook(file);
  WritableSheet sheet = book.createSheet("第一页", 0);

  Date currentTime = new Date();
  SimpleDateFormat formatter = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
  String dateString = formatter.format(currentTime);
  Label date = new Label(0, 0, "数据保存时间：" + dateString);
  ...
  try{
    sheet.addCell(date);
    book.write();
    book.close();
    }
    catch(jxl.write.WriteException e){
    }
  }
  catch(IOException e){
  }

具体的使用案例可以参考我的 Github project。

Java 串口编程

David Dong — Sun, 21 Oct 2012 18:55:59 +0800

前些天工作中用到了串口编程。Java 上位机 UI 需要通过串行端口访问外部的设备，Java 对串口设备的访问有提供库支持，使用起来比较容易，这里简单总结一下。

首先先下载javacomm20-win32的package。

下载链接

从解压包中有三个重要的文件,

文件	说明
comm.jar	提供了通讯用的java API
win32com.dll	提供了供comm.jar调用的本地驱动接口
javax.comm.properties	是这个驱动的类配置文件

然后把这三个文件放到如下目录中：

comm.jar可以放在JDK中的jre\lib\ext目录下,比如JAVA SDK装在C:\jdk1.6中，COMM.JAR放到C:\jdk1.6\jre\lib\ext;
win32com.dll可以直接放在JDK中的bin目录下，C:\jdk1.6\bin;
javax.comm.properties放在JDK中的jre\lib目录下，C:\jdk1.6\jre\lib;

使用上可以参考我写的如下代码。
CommBean.java 实现了一个针对串口的通用操作类，封装了串口设备的打开,配置,关闭，读写操作。

/*
 * To change this template, choose Tools | Templates
 * and open the template in the editor.
 */
package com.dong.serialcom;

import com.dong.factory.ConstantFactory;
import java.io.DataInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.util.Enumeration;
import java.util.TooManyListenersException;
import java.util.Vector;
import javax.comm.CommPortIdentifier;
import javax.comm.NoSuchPortException;
import javax.comm.PortInUseException;
import javax.comm.SerialPort;
import javax.comm.SerialPortEvent;
import javax.comm.SerialPortEventListener;
import javax.comm.UnsupportedCommOperationException;
/**
 *
 * @author Dong Gang
 */
public class ComBean implements SerialPortEventListener {

    static String portName;
    static int baudrate;
    static int dataBit;
    static int stopBit;
    static int parityBit;
    CommPortIdentifier portId;
    SerialPort serialPort;
    static OutputStream out;
    static InputStream in;
    private static ComBean serialComInstance = new ComBean();
    ReadThread readThread = new ReadThread();
    private byte[] dataPool = new byte[1024];
    private int[] b = new int[10240];
    private static int readBufferIndex = 0;
    private boolean readDataEnd = false;
    SerialBuffer RB;
    ReadSerial RT;
    private boolean isComProOver = false;
    private boolean isConfigureAndStart = false;
    private boolean isComTestOk = false;
    private boolean isComOpen = false;

    private ComBean() {

        portName = "";
        baudrate = 2000000;
        dataBit = SerialPort.DATABITS_8;
        parityBit = SerialPort.PARITY_NONE;
        stopBit = SerialPort.STOPBITS_1;

    }

    public static ComBean getInstances() {
        return serialComInstance;
    }

    public boolean closePort() {
        boolean rst = true;
        String infoStr = "端口关闭";
        if (isComOpen) {
            try {
                in.close();
                out.close();
                RT.stop();
                serialPort.notifyOnDataAvailable(false);
                serialPort.removeEventListener();
                serialPort.close();
                isComProOver = false;
                isConfigureAndStart = false;
                isComTestOk = false;

                if (ConstantFactory.timerRun != 0) {
                    ConstantFactory.timerRun = (int) 0;
                    ConstantFactory.timer.cancel();
                } else {
                }
            } catch (IOException e) {
                rst = false;
            }
        } else {
            rst = false;
        }

        return rst;
    }

    public String getPortName() {
        return portName;
    }

    public void setPortName(String portName) {
        ComBean.portName = portName;
    }

    public int getBaudrate() {
        return baudrate;
    }

    public void setBaudrate(int baudrate) {
        switch (baudrate) {
            case 0:
                ComBean.baudrate = 300;
                break;
            case 1:
                ComBean.baudrate = 600;
                break;
            case 2:
                ComBean.baudrate = 1200;
                break;
            case 3:
                ComBean.baudrate = 2400;
                break;
            case 4:
                ComBean.baudrate = 4800;
                break;
            case 5:
                ComBean.baudrate = 9600;
                break;
            case 6:
                ComBean.baudrate = 19200;
                break;
            case 7:
                ComBean.baudrate = 38400;
                break;
            case 8:
                ComBean.baudrate = 43000;
                break;
            case 9:
                ComBean.baudrate = 56000;
                break;
            case 10:
                ComBean.baudrate = 57600;
                break;
            case 11:
                ComBean.baudrate = 115200;
                break;
            case 12:
                ComBean.baudrate = 2000000;
            default:
                break;
        }
    }

    public int getDataBit() {
        return dataBit;
    }

    public void setDataBit(int dataBit) {
        switch (dataBit) {
            case 0:
                ComBean.dataBit = SerialPort.DATABITS_5;
                break;
            case 1:
                ComBean.dataBit = SerialPort.DATABITS_6;
                break;
            case 2:
                ComBean.dataBit = SerialPort.DATABITS_7;
                break;
            case 3:
                ComBean.dataBit = SerialPort.DATABITS_8;
                break;
            default:
                break;
        }
    }

    public int getParityBit() {
        return parityBit;
    }

    public void setParityBit(int parityBit) {
        switch (parityBit) {
            case 0:
                ComBean.parityBit = SerialPort.PARITY_NONE;
                break;
            case 1:
                ComBean.parityBit = SerialPort.PARITY_ODD;
                break;
            case 2:
                ComBean.parityBit = SerialPort.PARITY_EVEN;
                break;
            default:
                break;
        }
    }

    public int getStopBit() {
        return stopBit;
    }

    public void setStopBit(int stopBit) {
        switch (stopBit) {
            case 0:
                ComBean.stopBit = SerialPort.STOPBITS_1;
                break;
            case 1:
                ComBean.stopBit = SerialPort.STOPBITS_1_5;
                break;
            case 2:
                ComBean.stopBit = SerialPort.STOPBITS_2;
                break;
            default:
                break;
        }
    }

    public Vector getComId() {

        Vector comIdList = new Vector();
        Enumeration commId = CommPortIdentifier.getPortIdentifiers();
        CommPortIdentifier portId;

        while (commId.hasMoreElements()) {
            portId = (CommPortIdentifier) commId.nextElement();

            if (portId.getPortType() == CommPortIdentifier.PORT_SERIAL) {
                comIdList.add(portId.getName());
            }
        }
        return comIdList;
    }

    public boolean initCom() {

        ConstantFactory.infoStr = "";
        ConstantFactory.comProRun = false;
        isComOpen = false;
        try {
            portId = CommPortIdentifier.getPortIdentifier(portName);
            try {
                serialPort = (SerialPort) portId.open("Serial_Communication", 2000);
            } catch (PortInUseException e) {
                ConstantFactory.infoStr = "端口已经被占用！";
                return false;
            }
            try {
                in = new DataInputStream(serialPort.getInputStream());
                out = serialPort.getOutputStream();
            } catch (IOException e) {
                ConstantFactory.infoStr = "IO 错误！";
                return false;
            }
            try {
                serialPort.setSerialPortParams(baudrate, dataBit, stopBit, parityBit);
            } catch (UnsupportedCommOperationException e) {
                ConstantFactory.infoStr = "参数设置错误！";
                return false;
            }

            if (false) {
                try {
                    serialPort.addEventListener(this);
                } catch (TooManyListenersException e) {
                    serialPort.close();
                    ConstantFactory.infoStr = "监听错误！";
                    return false;
                }
                serialPort.notifyOnDataAvailable(true);
            }

            RB = new SerialBuffer(serialComInstance);
            RT = new ReadSerial(in, RB);
            RT.setPriority(Thread.MAX_PRIORITY);
            RT.start();

        } catch (NoSuchPortException e) {
            ConstantFactory.infoStr = "端口未找到！";
            return false;
        }
        isComOpen = true;
        return isComOpen;
        
    }

    public void serialEvent(SerialPortEvent e) {
        switch (e.getEventType()) {
            case SerialPortEvent.BI:
            case SerialPortEvent.OE:
            case SerialPortEvent.FE:
            case SerialPortEvent.PE:
            case SerialPortEvent.CD:
            case SerialPortEvent.CTS:
            case SerialPortEvent.DSR:
            case SerialPortEvent.RI:
            case SerialPortEvent.OUTPUT_BUFFER_EMPTY:
                break;
            case SerialPortEvent.DATA_AVAILABLE:
                readComm();
                break;
            default:
                break;
        }
    }

    void readComm() {
        dataAnalysis();
    }

    public void receiveData() {
        int k = 0, f = 0, s = 0, l = 0, y = 0;
        readDataEnd = false;
        try {
            b[readBufferIndex] = in.read();
            if (b[readBufferIndex] == 170) {
                readBufferIndex = 0;
                readDataEnd = true;
            } else {
                readBufferIndex++;
            }
  }
           
        } catch (IOException e) {
        }
    }

    public void dataAnalysis() {
        int c;
        try {
            c = in.read();
            b[readBufferIndex] = c;
            readBufferIndex++;
            if (c == 170) {
                int n = 0;
            }
            if (readBufferIndex == 6) {
                int m = 0;
            }
        } catch (IOException e) {
        }
    }

    class ReadThread extends Thread {

        public ReadThread() {
        }

        public void run() {
            while (!ConstantFactory.comProRun) {
                readCom();
            }
        }

        public void stopRun() {
            ConstantFactory.comProRun = false;
        }

        public void readCom() {
            byte[] dataByte = new byte[1024];
            try {
                while (in.available() > 0) {
                    int readnum = in.read(dataByte);
                }

            } catch (IOException e) {
            }
        }
    }

    public void writeCom(String wrtStr) {
        try {
            for (int i = 0; i < wrtStr.length(); i++) {
                out.write(wrtStr.charAt(i));
            }
        } catch (IOException e) {
        }
    }

    public void writeCom(byte[] wrtByte, int length) {
        try {
            for (int i = 0; i < length; i++) {
                out.write(wrtByte[i]);
            }
        } catch (IOException e) {
        }
    }

    public void writeCom(byte[] b) {
        try {
            out.write(b);

        } catch (IOException e) {
        }
    }

    public boolean getIsComProOver() {
        return isComProOver;
    }

    public void setIsComProOver(boolean isComProOver) {
        this.isComProOver = isComProOver;
    }

    public boolean getIsConfigureAndStart() {
        return isConfigureAndStart;
    }

    public void setIsConfigureAndStart(boolean isConfigureAndStart) {
        this.isConfigureAndStart = isConfigureAndStart;
    }

    public boolean getIsComTestOk() {
        return isComTestOk;
    }

    public void setIsComTestOk(boolean isComTestOk) {
        this.isComTestOk = isComTestOk;
    }
}

ReadSerial.java 读取类，使用 CommBean 的对象进行串口的数据读取。

/*
 * To change this template, choose Tools | Templates
 * and open the template in the editor.
 */
package com.dong.serialcom;

import java.io.IOException;
import java.io.InputStream;

/**
 *
 * @author Dong Gang
 */
public class ReadSerial extends Thread {

    private InputStream ComPort;
    private SerialBuffer Buffer;
 

    public ReadSerial(InputStream ComPort,SerialBuffer buffer) {
        this.ComPort = ComPort;
        this.Buffer = buffer;
    }

    public void run() {
        int c;
        try {
            while (true) {
               if(0<ComPort.available()){
                     c = ComPort.read();
                     Buffer.putByte(c);
                }
               
            }
        } catch (IOException e) {
        }
    }
}

我的Github上的项目链接如下 Github 项目。

Java 的线程和 Runnable 接口

David Dong — Fri, 01 Jun 2012 13:27:23 +0800

Java 实现多线程编程的方式有两种，一种是继承 Thread 类，另一种是实现 Runnable 接口。
下面的内容会分别介绍这两种用法以及使用上的区别。

Thread 类

Java 通过Thread类来实现多线程，关于 Thread 类的介绍，可以查阅 Java文档 Thread。
Thread 类继承自java.lang.Object，实现了 Runnable 接口。
Thread 类的用法如下。
MyThreadClass.java

package com.daviddong;

public class MyThreadClass extends Thread {

    private int cnt = 0;
    
    @Override
    public void run() {
        // TODO Auto-generated method stub
        cnt++;
        System.out.println("线程" + Thread.currentThread().getName() + 
        " 被调用 " + cnt + "次");
    }

}

App.java

package com.daviddong;
import java.util.ArrayList;

public class App 
{

    public static void main( String[] args )
    {
        ArrayList<MyThreadClass> lists = new ArrayList<MyThreadClass>();
        for(int i=0; i<10; i++){
            MyThreadClass tmp = new MyThreadClass();
            lists.add(tmp);
        }
        for(MyThreadClass tmp:lists){
            tmp.start();
        }
    }

}

运行结果

线程Thread-0 被调用 1次
线程Thread-9 被调用 1次
线程Thread-8 被调用 1次
线程Thread-7 被调用 1次
线程Thread-5 被调用 1次
线程Thread-6 被调用 1次
线程Thread-3 被调用 1次
线程Thread-4 被调用 1次
线程Thread-2 被调用 1次
线程Thread-1 被调用 1次

由以上结果可以看出，线程的运行结果与执行顺序无关，并且10个线程分别被执行一次，线程内资源无法共享。

Runnable 接口

Runnable 接口的 Java 文档的介绍。Runnable Runnable 接口内只声明一个方法Run()的方法，同 Thread 内的Run()方法一样，该方法内定义了线程的执行体。
Runnable 接口的用法如下。
MyRunnable.java

package com.daviddong;

public class MyRunnable implements Runnable{

    private int cnt = 0;
    @Override
    public void run() {
        // TODO Auto-generated method stub
        cnt++;
        System.out.println("线程" + Thread.currentThread().getName() + 
        " 被调用 " + cnt + "次");
    }

}

App.java

package com.daviddong;
import java.util.ArrayList;

public class App 
{

    public static void main( String[] args )
    {
        MyRunnable mR = new MyRunnable();
        ArrayList<Thread> lists = new ArrayList<Thread>();
        for(int i=0; i<10; i++){
            Thread tmp = new Thread(mR);
            lists.add(tmp);
        }
        for(Thread tmp:lists){
            tmp.start();
        }   
    }

}

运行结果

线程Thread-1 被调用 2次
线程Thread-2 被调用 3次
线程Thread-0 被调用 2次
线程Thread-7 被调用 7次
线程Thread-6 被调用 6次
线程Thread-5 被调用 5次
线程Thread-4 被调用 4次
线程Thread-8 被调用 9次
线程Thread-9 被调用 9次
线程Thread-3 被调用 10次

同 Thread 的例子一样，看的出线程的执行顺序与调用顺序无关。但是因为不同的线程中传入的是同一个实现了 Runnable 接口的对象，因此内部的变量被共享。由以上两个例子，可以看出使用 Thread 类和 Runnable 接口创建线程的不同主要是有2点。

因为 Java 只能实现单继承，但可以实现多个接口，所以使用继承 Thread 类的方法就会导致无法继承其他的类，而 Runnable 接口就可以避免继承的局限。
使用 Runnable 接口适合于资源的共享。

上面的程序还有一个问题，我们看到使用 Runable 接口时，虽然资源是被共享的，但是顺序不对，而且有重复出现的情况，正常应该是每个线程被调用时，调用次数加一，且应该顺序增加。造成这个问题的原因是因为不同的线程在执行都会访问共享的资源，而这些线程没有实现同步，共享资源会在一个线程调用的过程中被另外的线程改变。解决这个问题的话就要用到synchronized，使线程之间同步运行。我们修改一下代码。
MyRunnale.java

package com.daviddong;

public class MyRunnable implements Runnable{

    private int cnt = 0;
    @Override
    public synchronized void run() {
        // TODO Auto-generated method stub
        cnt++;
        System.out.println("线程" + Thread.currentThread().getName() + 
        " 被调用 " + cnt + "次");
    }

}

使用synchronized修饰run()方法，使得其在执行时与其他的线程同步。经过synchronized修饰后线程在执行run()方法时会确认是否有其他线程正在执行，如果有的话就加入队列等待，知道可以获得执行权。执行结果如下。

线程Thread-1 被调用 1次
线程Thread-3 被调用 2次
线程Thread-2 被调用 3次
线程Thread-7 被调用 4次
线程Thread-0 被调用 5次
线程Thread-9 被调用 6次
线程Thread-8 被调用 7次
线程Thread-6 被调用 8次
线程Thread-5 被调用 9次
线程Thread-4 被调用 10次

现在我们看到共享资源是按顺序被调用且没有重复。
最后在介绍一些 Thread 类的常用方法。

Return	Function	Description
Thread	currentThread	返回代码段当前被调用的线程
void	sleep()	在指定的毫秒数内让当前”正在执行的线程”休眠（暂停执行）
void	start()	启动当前线程
boolean	isAlive()	当前线程是否处于激活状态
String	getName()	返回当前线程的名字
long	getID()	返回当前线程的ID
void	setPriority()	设置优先级
boolean	isDaemon()	是否为守护进程

Javadoc 使用

David Dong — Sun, 22 Apr 2012 22:44:19 +0800

相信很多有经验的程序员都知道写注释的重要性，程序注释不仅仅是对代码进行说明，它其实也是代码本身组成的一部分。对于Java程序的开发者来说，更是如此。

注释已经变成了代码API接口文档的一部分。这是因为Java提供了一个工具Javadoc，能够从注释中抽取信息，直接生成API的帮助文档。文档的格式就同Java SDK官方API文档保持一致。这对于善于写代码却头疼写API文档的人来说简直是一款神器。

下面就简单的对Javadoc的使用做一个说明。使用Javadoc工具生成API文档时需要按照Java SDK的规定使用一系列的标签来完成，需要使用标签的地方有，

类（接口）的注释
构造函数的注释
方法的注释
全局变量的注释
字段/属性的注释
对外提供的接口注释
在循环和逻辑分支组成的代码中的注释
需要特殊说明的地方的注释

@author (classes and interfaces only, required)
@version (classes and interfaces only, required. See footnote 1)
@param (methods and constructors only)
@return (methods only)
@exception (@throws is a synonym added in Javadoc 1.2)
@see
@since
@serial (or @serialField or @serialData)
@deprecated (see How and When To Deprecate APIs)

其中，

@author name-text

使用@Author选项时，将具有指定名称文本的Author条目添加到生成的文档中。文档注释可以包含多个@author标记。可以为每个@author标记指定一个名称，也可以为每个标记指定多个名称。

@deprecated deprecated-text

从JDK 5.0开始，可以使用@Deprecated指示不应再使用此API（即使它可能继续工作）。Javadoc工具将不推荐的文本移到主描述前面，将其放在斜体中，并在其前面加上一个粗体警告：“不推荐”。此标记在所有文档注释中都有效：overview、package、class、interface、constructor、method和field。

@version

对类的说明，标明该类模块的版本。默认为1.0

{@code text}

等同于{@literal}. 将文本标记为代码样式的文本，一般在Javadoc中只要涉及到类名或者方法名，都需要使用@code进行标记。

{@docRoot}

表示从任何生成的页到生成的文档（目标）根目录的相对路径。

@exception class-name description

按照官方文档解释：是@throws同义词，没有任何区分。具体说明在@throws里再讲。

{@link package.class#member label}

插入带有可见文本标签的内联链接，该链接指向引用类的指定包、类或成员名称的文档。此标记在所有文档注释中都有效。这个标记非常类似于@see–两者都需要相同的引用，并且接受完全相同的语法包类成员和标签。主要的区别是{@link}生成一个内联链接，而不是将链接放在“See Also”部分。

@value value-text

用于标注在常量上，{@value} 用于表示常量的值。

@param parameter-name description

后面跟参数名，再跟参数描述，对方法参数的描述。只对方法的注释有效。

@return description

该标签标记方法的返回值，只对方法的注释有效。

@see reference

@see 一般用于标记该类相关联的类,@see即可以用在类上，也可以用在方法上。

@serial field-description | include | exclude

用于默认可序列化字段的文档注释中。可选字段描述应解释字段的含义并列出可接受的值。

@serialData data-description

data-description数据描述以序列化形式记录数据的类型和顺序。这个标签很少使用，目前我基本上没有用到过。

@since since-text

根据官方文档解释，@since 表达的是被标记元素是哪个发布版本引入的，一般后面跟版本号，也可以跟是一个时间，表示文件当前创建的时间。该标签对所有Javadoc注释类型有效。

@throws class-name description

异常描述 , 用于描述方法内部可能抛出的异常。@throws和@exception标记是同义词。在生成的文档中添加一个Throws标签，其中包含类名和说明文本。类名是该方法可能引发的异常的名称。此标签仅在方法或构造函数的文档注释中有效。对于相同或不同的异常，可以在给定的文档注释中使用多个@throws标记。为了确保记录所有选中的异常，如果throws子句中的异常不存在@throws标记，Javadoc工具会自动将该异常添加到HTML输出中（没有描述），就好像它是用@throws标记记录的一样。

和 The Java API Documentation Generator.

命令

Javadoc的命令行语法如下：

    javadoc [ options ] [ packagenames ] [ sourcefiles ] [ @files ]

参数可以按照任意顺序排列。下面分别就这些参数和相关的一些内容进行说明：

Packagenames 包列表。这个选项可以是一系列的包名（用空格隔开），例如java.lang java.lang.reflect java.awt。不过，因为javadoc不递归作用于子包，不允许对包名使用通配符；所以你必须显示地列出希望建立文档的每一个包。
Sourcefiles 源文件列表。这个选项可以是一系列的源文件名（用空格隔开），可以使用通配符。javadoc允许四种源文件：类源代码文件、包描述文件、总体概述文件、其他杂文件。
类源代码文件：类或者接口的源代码文件。
包描述文件：每一个包都可以有自己的包描述文件。包描述文件的名称必须是package.html，与包的.java文件放置在一起。包描述文件的内容通常是使用HTML标记写的文档。javadoc执行时将自动寻找包描述文件。如果找到，javadoc将首先对描述文件中<body> </body>之间的内容进行处理，然后把处理结果放到该包的Package Summary页面中，最后把包描述文件的第一句（紧靠<body>）放到输出的Overview summary页面中，并在语句前面加上该包的包名。
总体概述文件：javadoc可以创建一个总体概述文件描述整个应用或者所有包。总体概述文件可以被任意命名，也可以放置到任意位置。
其他文件：这些文件通常是指与javadoc输出的HTML文件相关的一些图片文件、Java源代码文件（.java）、Java程序（.class）、Java小程序（Applets）、HTML文件。这些文件必须放在doc-file目录中。

Java 的 Iterable 接口使用

David Dong — Sun, 18 Mar 2012 13:27:23 +0800

Iterable<> 是 Java.lang 包中定义的一个接口, 根据 Java 文档的介绍该接口主要声明了一个Iterator()方法，该方法返回一个Iterator<>接口。

Java 文档 - Iterable

那我们接下来看一看Iterator<>接口主要内容，Iterator<>接口的 Java 文档里说明 Iterator<>接口主要声明了三个方法。

Java 文档 - Iterator

hasNext()
next()
remove()

看到这里就很清楚了，实现了Iterator<>接口的类可以进行迭代遍历，Iterator<>接口通过对以上3个方法的封装实现了对遍历访问的支持。Java里的集合类都实现了Iterator<>接口。

一个简单的应用例子。
Student.java - 基本的数据类

package com.daviddong.example.Iterator

public class Student {

    private String name;
    private int age;
    private String gender;
    
    public Student(String name, int age, String male) {
        this.name = name;
        this.age = age;
        this.gender = male;
    }
    
    public Student() {
    
        try {
    
        } catch (Exception e) { 
            //TODO: handle exception
        }
    
    }
    
    public String getName() {
        return name;
    }
    
    public void setName(String name) {
        this.name = name;
    }
    
    public int getAge() {
        return age;
    }
    
    public void setAge(int age) {
        this.age = age;
    }
    
    public String getGender() {
        return gender;
    }
    
    public void setGender(String gender) {
        this.gender = gender;
    }

}

MyIterableClass.java - 实现了Iterator接口

package com.daviddong.example.Iterator;
import java.util.ArrayList;
import java.util.Iterator;

public class MyIterableClass implements Iterator {

    private ArrayList<Student> am = new ArrayList<>();
    private int length;
    private int index;
    
    @Override
    public boolean hasNext() {
        // TODO Auto-generated method stub
        if(index<length){
            return true;
        }
        return false;
    }
    
    @Override
    public Student next() {
        // TODO Auto-generated method stub
        return am.get(index++);
    }
    
    public MyIterableClass(ArrayList<Student> am, int length, int index) {
        this.am = am;
        this.length = length;
        this.index = index;
    }

}

APP.java

package com.daviddong.example.Iterator

import java.util.ArrayList;
import java.util.Random;

public class App {

    public static void main(String[] args) {
    
        ArrayList<Student> list = new ArrayList<Student>();
        ArrayList<String> names = new ArrayList<String>();
        names.add("男");
        names.add("女");
        String gender;
        for (int i = 0; i < 10; i++) {
            int r = new Random().nextInt(100);
            gender = names.get(r%2);
            Student student = new Student("学生—" +i, (int)(Math.random()*10)+10, gender);
            list.add(student);
        }
    
        MyIterableClass myClass = new MyIterableClass(list, list.size(), 0);
    
        while (myClass.hasNext()) {
            Student stu = myClass.next();
            System.out.println(stu.getName() + " 年龄:" + stu.getAge() + " 性别:" + stu.getGender());
        }
    }

}

运行结果如下。

学生—0 年龄:19 性别:女
学生—1 年龄:11 性别:男
学生—2 年龄:19 性别:女
学生—3 年龄:19 性别:女
学生—4 年龄:18 性别:女
学生—5 年龄:13 性别:男
学生—6 年龄:17 性别:女
学生—7 年龄:19 性别:女
学生—8 年龄:12 性别:男
学生—9 年龄:14 性别:女

有人可能会问，为什么不直接将hasNext()，next()方法放在Iterable接口中，其他类直接实现就可以了？

原因是有些集合类可能不止一种遍历方式，实现了 Iterable 的类可以再实现多个 Iterator 内部类，通过返回不同的 Iterator 实现不同的遍历方式，这样更加灵活。如果把两个接口合并，就没法返回不同的 Iterator 实现类了。

Java 的 finalize() 方法总结

David Dong — Sun, 04 Dec 2011 23:42:04 +0800

Java 提供的finalize()方法是定义 JObject 类中的方法。作用是帮助我们进行资源释放，类似于 C++ 中的析构函数。但是需要注意的是该方法根本上和 C++ 中的析构函数是不同的。

我们都知道 Java 是运行在 JVM 上的，JVM 的垃圾回收机制承担了系统资源释放的任务，因此在编写Java 程序的时候是不用考虑显式的调用方法进行资源释放的。这是因为 Java 是完全面向对象的语言，所有对象都驻留在堆内存，因此局部对象就不存在。而与 Java 不同，C++ 支持局部对象（基于栈）和全局对象（基于堆）。因此在 C++ 语言中引入了析构函数用来处理资源的释放。析构函数的执行时机是确定的，在对象的作用域结束或者显式调用的时候会被执行。一般在 C++ 编程中，子类的析构函数都会重写父类的析构函数用于本类的资源的释放。

而 Java 的finalize()方法则不同，finalize()虽然也会被用来进行资源的释放，但是 finalize() 方法的执行时机确是不确定的。原因是因为finalize()是在垃圾回收时被调用的，而何时进行垃圾回收是由JVM 根据当前的资源状况来决定，因此这个时间是无法保证的。

通常 JVM 会对当前的内存状况进行评估，只有当资源快耗尽或者有进程因为资源不足退出和排队等待的时候，JVM 会进行垃圾的回收。这个时候finalize()会被执行。因此如果我们想在确定的时间点用finalize()来释放有关的资源是十分不可靠的。

有意思的一点，Java 的垃圾回收机制虽然简化了编程，程序员无需考虑资源的释放导致的内容泄漏问题，但是却无法知道申请的对象何时被系统撤销，即使该对象的作用域是可以预知的（不再被运行状态引用或间接地通过其他对象引用时）。

当然，Java 也提供了System.gc()方法，System.gc()会强制要求系统进行垃圾回收，从而可以增加finalize()的调用几率，缺点是强制进行垃圾回收会影响系统的性能。

下面对finalize()方法来做一个总结：

finalize()方法可以被用来进行系统资源的释放;
系统在进行垃圾回收，准备释放对象所占用的内存空间之前会调用对象的finalize()方法;
因为系统进行垃圾回收的时机不确定，所以finalize()的执行时间是不确定的;
finalize()只会在对象内存回收前被调用一次。

Java 的异常处理

David Dong — Wed, 27 Jul 2011 20:02:36 +0800

想要理解Java的异常处理机制，只要掌握好这五个keywords 就可以了。
try, catch, finally, throw 和 throws

异常捕获

一个常用的异常捕获流程如下图所示，

try{
    需要监听异常的代码;
}catch(Exception e){
    异常处理代码;   
}finally{
    资源释放代码; //此段代码无论有无异常发生都会执行。
}

此外还有try-catch,try-finally的用法，是try-catch-finally的不同功能的组合。我们也可以在try的后面跟若干个catch，用于捕获多个异常。值得注意的是，无论有无执行到catch语句 (捕获异常) finally里面的代码一定会被执行, 执行的时间点是在try或者catch的return或者throw语句之前。也就是说在程序退出前会执行到finally包括的语句，然后回来执行try或者catch块中的return或者throw语句。如果finally中使用了return或者throw等终止方法的语句，则就不会跳回执行，直接停止。如果finally块中抛出异常则会覆盖前面try或者catch块中的异常。

throw 用来主动抛出异常。
throws 用在方法的声明当中，如果该方法声明throws一个异常，则表明该方法里可以不处理该异常，有JVM将异常传递里上层调用者处理。 throw通常和throws联合使用，抛出的是程序中已经产生的异常类实例。

异常分类

看完了异常处理的流程，我们再来看看Java异常的分类。 Java的异常都来源于一个接口throwable，throwable是java.lang包里的一个接口。继承java.lang.Object.其中有两个直接子类ERROR和EXCEPTION，有兴趣可以参考 Java 官方文档
Throwable 主要提供了如下的一些方法。
Throwable getCause()
String getMessage()
void printStackTrace()
String toString()

ERROR是指的系统的错误，用户无法处理，比如内存溢出等，只有通过修改程序来更正，因此遇到此类错误会让程序终止运行。
ERROR和子类的关系如下图所示。

EXCEPTION指的是运行时的异常，是用户可以进行处理的异常。
EXCEPTION类和子类的关系如下。

Exception 和 RuntimeException

可以看到RuntimeException是Exception的子类，实际上RuntimeException对应的是非检查性异常，用户可以处理也可以不处理，而如何继承的是exception,则为检查性异常，用户必须用try-catch来处理异常。
一个简单的例子。

/**
 * example of java exception
 *
 */
public class App 
{
    public static void main( String[] args )
    {
        runTimeExFun();
    }

    private static void runTimeExFun() throws RuntimeException{

        throw new RuntimeException("new runtime exception!");
        
    }
}

运行结果

Exception in thread "main" java.lang.RuntimeException: new runtime exception!
        at daviddong.example.exception.App.runTimeExFun(App.java:16)
        at daviddong.example.exception.App.main(App.java:11)

可以看出如果函数声明的是抛出一个RuntimeException异常，则主调函数可以选择不用try-catch来处理。我们将throws的异常改为Exception,

/**
 * example of java exception
 *
 */
public class App 
{
    public static void main( String[] args )
    {
        runTimeExFun();
    }

    private static void runTimeExFun() throws Exception{
        throw new RuntimeException("new runtime exception!");
    }
}

则会报语法错误“Unhandled exception type ExceptionJava(16777384)”，需要用try-catch来处理异常。
正确代码如下。

/**
 * example of java exception
 *
 */
public class App {
    public static void main(String[] args) {
        try {
            runTimeExFun();
        } catch (Exception e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        }
    }

    private static void runTimeExFun() throws Exception{
        throw new RuntimeException("new runtime exception!");
    }
}

Java 里的引用和 C++ 指针的区别

David Dong — Sat, 23 Apr 2011 23:18:23 +0800

本文对 Java 里的引用的概念做一个简单的总结，不做过多的阐述。

Java 里面除了基本数据类型外,其他所有对象的引用都是指针引用.基本数据类型也可以用对象来引用。所谓指针引用就是传递的是对象的地址,而不是该对象值的拷贝。

因此从这一点上来说 Java 的引用和 C++ 里面的指针的作用是一致的。
Java 里的引用和 C++ 的区别是 C++ 里的指针类型变量可以做运算,强制类型转换。 Java 里面的引用避免了这个问题.
Java 语言的安全性高一方面是所有的引用都是对象引用,没有指针的概念。另一个方面自动垃圾回收机制,避免了内存泄露。
Java 的基本数据类型及其封装类如下。

数据类型	封装类
byte	Byte
char	Character
double	Double
float	Float
int	Integer
long	Long
short	Short
boolean	Boolean

这些基本类型的封装类提供了一些方法，例如 toString(), 和包括与其他类型之间的相互转换等。

Java 的序列化(Serializable)

David Dong — Sat, 12 Feb 2011 20:02:36 +0800

Java的序列化提供了一种保存对象状态的机制，之所以要用到序列化是因为有时我们会在对象的生命期结束后需要把对象状态保存下来，通常是存放到外部的媒介上，比如文件，磁盘或网络上。并且在需要时能够通过一种机制来恢复。序列化能够在不同的JVM之间共享数据。

实现类的序列化在语法上非常简单，只需要实现Serializable接口即可。Serializable接口没有需要实现的方法（这是一个”标记接口-tagging interface”)，只是标明该类的对象需要序列化。

通常我们在构建javabean的时候，定义Bean时要实现这个接口，因为Bean的属性状态一般都是需要保存的。

关于序列化

需要注意以下几点：

序列化保存的是对象的状态。状态包括对象的非静态成员变量（包括声明为private的变量），不能保存任何的成员方法和静态的成员变量。
当一个父类实现序列化时,子类自动序列化,不需要显式实现Serializable接口。
当一个对象的实例变量引用其他对象,序列化该对象时也把引用对象序列化。
序列化的对象包括基本数据类型，所有集合类以及其他许多东西，还有对象。但是并非所有的对象都可以序列化。
对象序列化不仅保存了对象的状态，而且还能保存对象内包含的所有引用的对象状态，一直追踪知道所有的引用的对象被保存。
使用transient关键字修饰的的变量，在序列化对象的过程中，该属性不会被序列化。常用于当某些变量不想被序列化，同是又不适合使用static关键字声明时。

关于 serialVersionUID

序列化运行时使用一个称为serialVersionUID的版本号与每个可序列化类相关联，该序列号在反序列化过程中用于验证序列化对象的发送者和接收者是否为该对象加载了与序列化兼容的类。如果接收者加载的该对象的类的serialVersionUID与对应的发送者的类的版本号不同，则反序列化将会导致InvalidClassException。可序列化类可以通过声明名为 serialVersionUID的字段（该字段必须定义为 static final long）显式声明其自己的serialVersionUID.建议在一个可序列化类中显示的定义serialVersionUID。因为serialVersionUID的取值是Java运行时环境根据类的内部细节自动生成的。如果对类的源代码作了修改，再重新编译，新生成的类文件的serialVersionUID的取值有可能也会发生变化。而且不同的Java编译器之间会有差异。可以用在如下的场合:

在某些场合，希望类的不同版本对序列化兼容，因此需要确保类的不同版本具有相同的serialVersionUID；
在某些场合，不希望类的不同版本对序列化兼容，因此需要确保类的不同版本具有不同的serialVersionUID

参考代码如下，改代码演示了如何序列化和反序列化一个对象：
首先定义一个类实现Serializable接口。
DemoInstance.java

package Serializable;
import java.io.Serializable;
public class DemoInstance implements Serializable{

   /**
    * 定义一个明确的 serialVersionUID 而不是由系统自动产生, 声明为 static final long 类型 
   */
    private static final long serialVersionUID = 1L;

    private String a;
    public String b;
    protected String c;

    public String getA() {
        return a;
    }

    public void setA(String a) {
        this.a = a;
    }

    public String getB() {
        return b;
    }

    public void setB(String b) {
        this.b = b;
    }

    public String getC() {
        return c;
    }

    public void setC(String c) {
        this.c = c;
    }

}

在main()中实现该类对象的序列化和反序列化。
App.java

package Serializable;

import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.ObjectInputStream;
import java.io.ObjectOutputStream;

/**
 * example of serializable
 *
 */
public class App {
    public static void main(String[] args) {

        /* 声明一个对象，该对象实现了serializable接口 */
        DemoInstance demoObj = new DemoInstance();
        DemoInstance demoReadObj = new DemoInstance();

        demoObj.setA("I am private member.");
        demoObj.setB("I am protected member.");
        demoObj.setC("I am public member.");

        demoReadObj.setA("I am null!");
        demoReadObj.setB("I am null!");
        demoReadObj.setC("I am null!");

        try {
            /*首先声明一个FileOutputStream的对象，用于操作要写入的文件*/
            FileOutputStream fos = new FileOutputStream("example_Serializable.txt");
            /* 声明一个ObjectOutputStream 对象，将要写入的文件流与要存的对象关联 */
            ObjectOutputStream oos = new ObjectOutputStream(fos);
            /* 写入要保存的对象 */
            oos.writeObject(demoObj);
            oos.close();

            /* 首先声明一个FileInputStream的对象，用于操作要写入的文件 */
            FileInputStream fis = new FileInputStream("example_Serializable.txt");
            /* 声明一个ObjectInputStream 对象，将要读入的文件流与要存的对象关联 */
            ObjectInputStream ois = new ObjectInputStream(fis);
            demoReadObj = (DemoInstance) ois.readObject();
            ois.close();    

            System.out.println("A:"+demoReadObj.getA()
            +"\nB:"+demoReadObj.getB()+"\nC:"+demoReadObj.getC());


        } catch (IOException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        } catch (ClassNotFoundException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        }

    }
}

以上代码实现了对象的序列化，注意此时如果将保存的文件用文本编辑器打开会看到乱码。
这是因为序列化和反序列化都是基于二进制流的，将对象保存的信息转化为二进制存储在了文件中，那么用文本编辑器打开查看的话当然是会出现乱码的。只有通过反序列化才能将存储的二进制读取出来。

反序列化读取的代码如下。

    /* 首先声明一个FileInputStream的对象，用于操作要写入的文件 */
    FileInputStream fis = new FileInputStream("example_Serializable.txt");
    /* 声明一个ObjectInputStream 对象，将要读入的文件流与要存的对象关联 */
    ObjectInputStream ois = new ObjectInputStream(fis);
    demoReadObj = (DemoInstance) ois.readObject();
    ois.close();

结果如下所示。

A:I am private member.
B:I am protected member.
C:I am public member.

以上是序列化的基本步骤，下面我们再定义一个DemoInstance的子类，该类没有直接实现Serializable接口，同时在该类中还引用了另外一个没有实现Serializable接口的实体类。
SubDemoInstance.java

package Serializable;

public class SubDemoInstance extends DemoInstance {

    /**
     * 定义一个明确的 serialVersionUID 而不是由系统自动产生, 声明为 static final long 类型
     */
    private static final long serialVersionUID = 2L;

    private String state;
    /* 引用一个没有序列化的对象 */
    private ClassWithoutSerial obj = new ClassWithoutSerial();

    public void setPara(String state) {

        this.obj.setState(state);
    }

    public void setPara(ClassWithoutSerial obj, String state) {

        obj.setState(state);
    }

    public String getDeclare() {
        return state;
    }

    public void setDeclare(String state) {
        this.state = state;
    }

    public ClassWithoutSerial getObj() {
        return obj;
    }

    public void setObj(ClassWithoutSerial obj) {
        this.obj = obj;
    }
    
}

ClassWithoutSerial.java

package Serializable;

/* 类没有实现Serializale接口 */
public class ClassWithoutSerial {

    private String state;

    public String getState() {
        return state;
    }

    public void setState(String state) {
        this.state = state;
    }

    public ClassWithoutSerial(String state) {
        this.state = state;
    }

    public ClassWithoutSerial() {
    }

}

实现序列化和发序列化的代码。

    SubDemoInstance subDemoInstance = new SubDemoInstance();

    subDemoInstance.setA("I am private member in subclass.");
    subDemoInstance.setB("I am protected member in subclass.");
    subDemoInstance.setC("I am public member in subclass.");
    subDemoInstance.setDeclare("I am member in subclass");
    subDemoInstance.setPara(cws,"I am be called by subclass.");
    
    /*Serialize
     首先声明一个FileOutputStream的对象，用于操作要写入的文件 */
    FileOutputStream fosi = new FileOutputStream("example_Serializable.txt");
    /* 声明一个ObjectOutputStream 对象，将要写入的文件流与要存的对象关联 */
    ObjectOutputStream oosi = new ObjectOutputStream(fosi);
    /* 写入要保存的对象 */
    oosi.writeObject(subDemoInstance);
    oosi.close();

    /*de-serialize
      首先声明一个FileInputStream的对象，用于操作要写入的文件 */
    FileInputStream fisi = new FileInputStream("example_Serializable.txt");
    /* 声明一个ObjectInputStream 对象，将要读入的文件流与要存的对象关联 */
    ObjectInputStream oisi = new ObjectInputStream(fisi);
    SubDemoInstance readSubDemo =  new SubDemoInstance();
    readSubDemo = (SubDemoInstance) oisi.readObject();
    oisi.close();

    System.out.println("A:" + readSubDemo.getA() + "\nB:" + 
    readSubDemo.getB() + "\nC:" + readSubDemo.getC()
    +"\nSubClass:"+readSubDemo.getDeclare()
    +"others:"+readSubDemo.getObj().getState());

执行后会出现如下错误。

A:I am private member.
B:I am protected member.
C:I am public member.
java.io.NotSerializableException: Serializable.ClassWithoutSerial
    at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1184)
    at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1548)
    at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1509)
    at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432)
    at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178)
    at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348)
    at Serializable.App.main(App.java:61)

可以看到错误为ClassWithoutSerial没有序列化却没执行了序列化的操作。稍作修改使得类ClassWithoutSerial实现Serializable接口，结果如下。

A:I am private member in subclass.
B:I am protected member in subclass.
C:I am public member in subclass.
SubClass:I am member in subclass
others:I am be called by subclass.

由此可见，序列化类的子类可以不必实现序列化，但是其内部引用的类对象必须实现序列化。

David Dong's Blog

Rust startup - trait

Rust startup - Ownership

The Ownership Rules

The Benefits of Ownership

Ownership and Borrowing

Mutable Borrowing

Clone

Ownership in Functions

The Ownership Checker

Lifetimes

Example

Rust startup - environment setup

Environments

Setup

Rust Compile environment and related tools installation

Vscode configuration for development of Rust

Vscode debugging Rust code

What it's like to use ChatGPT

About gerrit.createChangeId

Keep old change-id when do git commit

Git hook 配置与使用

Git Hooks 介绍

Husky 配置

–no-verify 的使用

Windows 消息处理机制

消息队列

流程如下

消息结构体

发送消息函数 SendMessage () 和 PostMessage () 的区别

读取消息函数 GetMessage () 和 WaitMessage (), PeekMessage ()

浅谈 Java 线程同步技术

synchronized

wait 和 notify

volatile

重入锁

原子类型变量

Git commands and usage tips

推送到远程分支

用法：git push <远程主机名> <本地分支名>:<远程分支名>

1. 将本地的 master 分支推送到远程 master 分支

2. 将本地的 dev 分支推送到远程 master 分支

3. 删除远程 master 分支

4. 添加远程主分支和本地 master 分支的绑定关系

5. 强制推送本地dev分支到远程master分支

6. 推送本地当前分支到远程 master分支

拉取远程分支到本地

用法：git pull <远程主机名> <远程分支名>:<本地分支名>

1. 拉取远程 master 分支到本地 dev 分支

2. 拉取远程 master 分支到本地当前分支

3. 建立本地分支与远程分支的追踪关系

4. git pull origin master:dev 等于

5. 采用 rebase 的方式合并本地分支

创建新分支

1. 创建 newbranch 分支

2. 创建新分支并与远程分支关联

删除分支

1. 删除本地分支

2. 强制删除本地分支

3. 删除远程 master 分支

重命名分支

添加远程分支

git remote add

克隆远程分支

git clone

查看远程提交记录

添加 Github Token 到本地仓库

git stash

应用场景

合并分支

git merge

1. merge dev 分支和 dev2 分支

2. 手动提交

3. 如果有冲突，处理冲突

git cherry-pick

1. 复制 dev2 的 aa3b6ca2 b821c4f7e1c 提交到 dev 分支

git rebase

用法：

1. 合并当前分支的多个提交记录

合并3ce944b8c..7f66117d96ac6 三条提交记录

1. 显示远程 dev 分支记录中有关 README.md 的提交