2010年11月29日 星期一

我的git使用經驗

我在一個新的sdk中自已用了git來測驗
我發現git-status有時候很慢,查了一下,有人說用git-gc來加快access
於是我下了git-gc,等了幾分鐘後還是沒run完,一看memroy,竟被吃光了:

peter$ git-gc
Counting objects: 88839, done.

Compressing objects:  18% (15328/84733)
後來改用
git-diff --stat就快多了

petert$ git diff --stat drivers/
 .../hal/linux/public/mipsisa32-be-elf.opt_ah.h     |   18 +++---------------
 1 files changed, 3 insertions(+), 15 deletions(-)
git-status用git-commit同樣的options, 主要是用來比較git-commit時會發生那些變動
git-status其實就是git-commit --preview, 見意採用git-diff --stat…
http://stackoverflow.com/questions/715321/git-status-is-there-a-way-to-show-changes-only-in-a-specific-directory

The reason that git status takes the same options as git commit is that the purpose of git status is to show what would happen if you committed with the same options as you passed to git status. In this respect git status is really git commit --preview.
To get what you want, you could do this which shows staged changes:
git diff --stat --cached -- 
and this, which shows unstaged changes:
git diff --stat -- 
or this which shows both:
git diff --stat HEAD -- 

lighttpd porting


CC="mips-linux-gcc -I/tmp/rootfs/lighttpd/include -L/tmp/rootfs/lighttpd/lib" LD="mips-linux-ld -L/tmp/rootfs/lighttpd/lib" ./configure --target=mips-linux --host=mips-linux --without-zlib --prefix=/tmp/rootfs/lighttpd --without-bzip2 --disable-ipv6 --with-pcre=/tmp/rootfs/lighttpd/lib/

-----------------------------------------------------
設定 lighttpd.conf
  1. document root
    修改lighttpd.conf
    var.server_root = "/tmp/rootfs/www"
    server.port = 1234
    var.conf_dir    = "/tmp/rootfs/etc/lighttpd"
  2. cgi module.
    修改modules.conf
    include "conf.d/cgi.conf"
  3. 修改conf.d/cgi.conf
    cgi.assign                 = ( ".pl"  => "/usr/bin/perl",
                                 ".cgi" => "/tmp/rootfs/www/cgi/ssi",
                                   ".rb"  => "/usr/bin/ruby",
                                   ".erb" => "/usr/bin/eruby",
                                   ".py"  => "/usr/bin/python" )



 在target board edit 「/tmp/rootfs/www/cgi/ssi」如下,and chmod +x.
#!/bin/sh
echo -ne "Content-Type: text/plain\n\n";
echo -ne "Hi there! This is a sample perl program!!!\n";
set

 測試 url: http://172.21.46.138:1234/index.cgi
CONTENT_LENGTH='0'
DOCUMENT_ROOT='/tmp/rootfs/www'
GATEWAY_INTERFACE='CGI/1.1'
HTTP_ACCEPT='text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8'
HTTP_ACCEPT_CHARSET='UTF-8,*'
HTTP_ACCEPT_ENCODING='gzip,deflate'
HTTP_ACCEPT_LANGUAGE='zh-tw,en-us;q=0.7,en;q=0.3'
HTTP_CACHE_CONTROL='max-age=0'
HTTP_CONNECTION='keep-alive'
HTTP_HOST='172.21.46.138:1234'
HTTP_KEEP_ALIVE='115'
HTTP_USER_AGENT='Mozilla/5.0 (Windows; U; Windows NT 5.1; zh-TW; rv:1.9.2.10) Gecko/20100914 Firefox/3.6.10 GTB7.1 (.NET CLR 3.5.30729)'
IFS='  
'
LD_LIBRARY_PATH='/tmp/rootfs/lighttpd/lib'
PATH='/usr/local/bin:/usr/bin:/sbin:/bin'
PPID='797'
PS1='\w \$ '
PS2='> '
PS4='+ '
PWD='/tmp/rootfs/www'
REDIRECT_STATUS='200'
REMOTE_ADDR='172.21.33.212'
REMOTE_PORT='2161'
REQUEST_METHOD='GET'
REQUEST_URI='/index.cgi'
SCRIPT_FILENAME='/tmp/rootfs/www/index.cgi'
SCRIPT_NAME='/index.cgi'
SERVER_ADDR='0.0.0.0'
SERVER_NAME='172.21.46.138'
SERVER_PORT='1234'
SERVER_PROTOCOL='HTTP/1.1'
SERVER_SOFTWARE='lighttpd/1.4.28'
當IE brower URL:http://172.21.46.138:1234/index.cgi 時,lighttpd會根據cgi.conf所設定的副檔名來執行程式(以此為例,真正的執行檔為「/tmp/rootfs/www/cgi/ssi, 而index.cgi這個file實際上可能不存在

但這不是我想要的結果:
所以我做了點小修改: 
  1. 將www/cgi/ssi link到www
    再將原來的ssi rename為ssi.exe
  2. 小改一下cgi.conf
    cgi.assign                 = ( ".pl"  => "/usr/bin/perl",
                                   ".cgi" => "/tmp/rootfs/www/cgi/ssi.exe",
                                   ".asp" => "/tmp/rootfs/www/cgi/ssi.exe",
                                   ".rb"  => "/usr/bin/ruby",
                                   ".erb" => "/usr/bin/eruby",
                                   ".py"  => "/usr/bin/python" )
peter@team-server:www$ tree cgi
cgi
|-- ssi -> ../
`-- ssi.exe


測試一下

http://172.21.46.138:1234/cgi/ssi/index.asp
可以work了!




2010年10月13日 星期三

CVS,GIT,Mercurial和SVN

轉自: http://hi.baidu.com/tigerwooz/blog/item/edbb40c64442ce159c163d74.html


最近关于DVCS讨论的如火如荼,转一篇4个著名VCS的比较
特征CVSGitMercurialSubversion
是否原子提交CVS: 没有. CVS提交不是原子的Git: 是的. 提交都是原子的Mercurial: 是的Subversion: 提交都是原子的
文件和目录是否可以移动或重命名CVS: 不是. 重命名不支持. 如果手动进行, 可能会损坏历史记录Git: 支持重命名, 这是很实用的目的. git甚至能检测到重命名之后文件的改变. 尽管如此, 基于特殊的存储结构, 重命名不会被显示的记录, git能够推导出来(在实际使用中很容易做到)Mercurial: 是的, 重命名是支持的Subversion: 是的. 支持重命名
在移动或重命名之后智能合并CVS: 不能. 重命名都不支持, 就不必说智能了Git: 不支持. 细节在Git FAQ里: “Git有一个重命名的命令git mv, 但是这仅仅是为了便利. 效果和移掉某个文件, 增加另外一个文件没有任何区别”Mercurial: 是的. 重命名之后智能合并是支持的. Mercurtial文档说:“如果我修改一个文件,而你重新命名了这个文件, 然后我们合并我们的变更, 那么我所做的修改就会被更新到根据旧文件名字而产生的新文件里(这可能就是你所期望的‘最简单的动作’, 但是不是所有版本控制系统都支持)Subversion: 不支持. “svn help me“中提到“注意: 这个子命令相当于拷贝和删除.“并且可能有个bug
文件和目录拷贝CVS: 不能. 拷贝不支持Git: 不能. 拷贝不支持Mercurtial: 是的. 支持拷贝Subversion: 是的. 并且拷贝非常容易(O(1)). 包括产生分支
远程存储仓库的备份CVS: 间接的. 可以使用John Polstra写的CVSupGit: 是的. 是git的内部特征Mercurial: 是的Subversion: 间接的. 可以使用Chia-liang Kao的SVN::Mirror插件(好像是台湾人)或Shlomi Fish的SVN-Pusher工具
是否传递变更到父仓库CVS: 不会Git: 是的(Linux内核开发过程经常使用这个特征)Mercurtial: 是的Subversion: 是的, 使用要么是Chia-Ling Kao的SVN::Mirror脚本或者Shlomi Fish的svn-push工具
仓库权限CVS: 很有限. “pre-commit hook scripts“能够被用来实现各种权限控制系统Git: 请看和Git一起附带的contrib/hooks/update-paranoid. 看和svnperms类似的path_rules的代码Mercutial: 是的. 它能够锁住仓库, 子目录或者使用hooks后的文件Subversion: 是的. 基于HTTP权限的WebDAV-based模块能够支持基于目录级的仓库
变更集CVS: 不是. 变更是基于文件的Git: 是的. 是支持的, 创建他们很容易Mercurial: 是的. 变更集是支持的Subversion: 部分支持. 对于一次提交会隐式创建一个变更集
跟踪线性的文件历史CVS: 是的. cvs annotateGit: 是的.(git blame)Mercurial: 是的(hg annotate)Subversion: 是的(svn blame)
能够只在仓库的单目录下作用CVS: 是的Git: 不是. 尽管如此, 提交多少能被限制, 请看“Repository Permissions”Mercurial: 能够基于某树的某个子集进行提交. 也有局部检出的能力Subversion: 是的
跟踪未提交的变化CVS: 是的. 通过cvs diffGit: 是的. 另外, 分支在git里非常智能, 在某些工作流里能够被当成是另外一个未提交代码的存储库. 请看“git stash“命令Mercurial: 是的. 使用hg diffSubversion: 是的. 使用svn diff
基于单个文件的提交信息CVS: 不是. 提交信息是基于单次变化的Git: 是的. 提交信息基于变更集Mercurial: 不是Subversion: 不是. 没有这个特征
文档CVS: 非常棒. 有很多在线的tutorials和资源, 在线的书籍. 命令行客户端也支持一个在线的帮助系统Git: 良好. 短的帮助比较简洁难懂. man页很有分量, 但容易误解. 有很多tutorialMercurial: 很好. 有基于公司的书籍和wiki. 每个命令都集成了帮助Subversion: 很好. 有一些在线的书籍和一些在线的tutorials和资源. 并且书籍是以docbook/xml写的所以很容易变换成其他格式. 命令行同样提供了在线的帮助系统
配置是否轻松CVS: 好. 是个事实上的标准. 基于每个系统都有并且很容易配置Git: 好. 在现有平台上二进制可用. 需要C编译器和Perl. 在windows上需要cygwin. 并有一些Unix特征Mercurial: 非常好. 几乎所有平台都有二进制包. 从源码编译需要python2.3以上, 并且需要C编译器Subversion: Subversion服务器需要安装在apache2模块里(如果有人希望HTTP作为底层协议的话)或使用它自身的服务器. 客户端需要Subversion特征的逻辑还有WebDAV库(针对HTTP). 安装组件很直接, 但是需要一些额外的工作(假定subversion在某些平台没有二进制包可用)
命令集CVS: 包含了3个经常用到的命令的简单的命令集(cvs commit, cvs update和cvs checkout)和其它一些Git: 命令集很丰富, 并且和CVS不兼容Mercurial: 尝试模仿CVS交互方式, 但是偏离了基于不同的设计的意图Subversion: 类CVS的命令集, 能够很容易被CVS用户使用
网络支持CVS: 好. cvs在不同的场合使用不同的协议. 协议能够通过ssh链接的加密隧道进行Git: 非常棒. 能够使用本地的git协议, 但也能在rsync, ssh, HTTP和HTTPS上使用Mercurial: 非常棒. 使用HTTP或ssh. 远程访问会非常安全, 在只读网络里不需要上锁Subversion: 非常好. Subversion服务器支持WebDAV+DeltaV(基于HTTP或HTTPS)作为底层协议, 或者它自身的协议同样能在ssh链接通道里使用.
可移植性CVS: 好. 客户端能在UNIX, Windows和Mac OS上使用. 服务器端能在UNIX, 附有UNIX模拟层的Windows上使用Git: 客户端运行在大多数的UNIX系统上, 但没有MS-Windows本地程序. 基于cygwin的系统看起来也能使用Mercurial: 非常棒. 运行在基于所有能运行python的平台.仓库是兼容性的基于CPU结构和字节序的Subversion: 非常好. 客户端和服务器端都能在UNIX, Windows和Mac OS X上运行
web接口CVS: 是的. CVSweb, ViewVC, Chora和wwCVSGit: 是的. Gitweb包含在发布包中Mercurial: 是的. Web接口是内置组件Subversion: 是的. ViewVC, SVN::Web, WebSVN, ViewSVN, mod_svn_view, Chora, Trac, SVN::RaWeb::Light, SVN Browser, Insurrection和perl_svn.另外, Subversion的apache服务也提供了一个基础的web接口
图形用户界面CVS: 非常好. 有很多图形界面可以用: WinCVS, Cervisia(对于KDE), TortoiseCVS(Windows浏览器插件)Git: Gitk包含在发行版中. Qqit和Git-gui工具也可使用Mercurial: 通过hgit扩展查看历史; 检入扩展(hgct)使得提交很容易. 一些第三方的IDEs和GUI工具(如eric3, meld)有一些集成的Mercurial支持Subversion: 非常好. 有很多GUIs可用: RapidSVN(跨平台), TortoiseSVN(Windows浏览器插件), Jsvn(java), 等. 大多数都还在开发中

2010年9月13日 星期一

iproute2 xfrm state command

這是我用來手動新增(test) ipsec的方法,當然還要配合policy
ip xfrm state add \
   src 172.21.46.133 dst 172.21.33.235 \
   proto esp spi 0xc1764476 mode tunnel \
   auth md5 0xbde367f6722286d104212c864d3041ff \
   enc blowfish \
 0xb860a53627435c27c5862ebf7e2e2de15e2fd2786db612eff0ce4b181a9373c34dd1b6ca17b1ffb2d118a575b9f8a9aa865783500b2cd38a

ip xfrm stat delete src 172.21.46.133 dst 172.21.33.235 proto esp spi 0xc1764476

 也可以用sha1加aes
ip xfrm state add src 172.21.46.131 dst 172.21.33.235  proto esp spi 0x4db377c8 reqid 16385 mode tunnel  auth sha1 0xe0a21cb441a0790188f8d5a7573608ae1e75af07 enc aes 0x86e4685642435deb2b53bee6c23723974d0207a4abed037c5028a310ef355a6e

ip xfrm state delete src 172.21.46.131 dst 172.21.33.235  proto esp spi 0x4db377c8
twofish
ip xfrm state add src 172.21.46.131 dst 172.21.33.235  proto esp spi 0x4db377c8 reqid 16385 mode tunnel  auth sha1 0xe0a21cb441a0790188f8d5a7573608ae1e75af07 enc twofish 0x86e4685642435deb2b53bee6c23723974d0207a4abed037c5028a310ef355a6e

ip xfrm state delete src 172.21.46.131 dst 172.21.33.235  proto esp spi 0x4db377c8
IPSec manual mode by iproute2
ip xfrm st add src 172.21.46.131 dst 172.21.33.235 proto esp spi 0x12345678 mode tunnel auth md5 0xbde367f6722286d104212c864d3041ff  enc aes 0xf82bbcccc0e01308e9a8edba1f2c058be3af44bfde5c26657d4a6609ca488ac2

----------------------------------------------------------------------------------------------

其它可以參考的url
http://lwn.net/Articles/375829/
http://osdir.com/ml/linux.kernel.cryptoapi/2008-04/msg00017.html

[lho@svdclab161 sec]$ cat ip-start-transport-ccm
#!/bin/sh

NODE=$1

echo "Starting IPSec transport mode using CCM..."

./ip xfrm policy flush
./ip xfrm state flush
#
# SA
./ip xfrm state add src 10.66.21.164 dst 10.66.21.166 proto esp spi
0x201 mode transport aead "rfc4309(ccm(aes))"
0x0102037aeaca3f87d060a12f4a4487d5a5c335 96
./ip xfrm state add src 10.66.21.166 dst 10.66.21.164 proto esp spi
0x301 mode transport aead "rfc4309(ccm(aes))"
0x010203f6ddb555acfd9d77b03ea3843f265325 96
#
# Policy
if [ "${NODE}" = "A" ]; then
./ip xfrm policy add dir out src 10.66.21.164 dst 10.66.21.166
tmpl proto esp mode transport
./ip xfrm policy add dir in src 10.66.21.166 dst 10.66.21.164
tmpl proto esp mode transport
fi
if [ "${NODE}" = "B" ]; then
./ip xfrm policy add dir in src 10.66.21.164 dst 10.66.21.166
tmpl proto esp mode transport
./ip xfrm policy add dir out src 10.66.21.166 dst 10.66.21.164
tmpl proto esp mode transport
fi

[lho@svdclab161 sec]$ cat ip-start-transport-gcm
#!/bin/sh

NODE=$1

echo "Starting IPSec transport mode using GCM..."

./ip xfrm policy flush
./ip xfrm state flush
#
# SA
./ip xfrm state add src 10.66.21.164 dst 10.66.21.166 proto esp spi
0x201 mode transport aead "rfc4106(gcm(aes))"
0x010203047aeaca3f87d060a12f4a4487d5a5c335 96
./ip xfrm state add src 10.66.21.166 dst 10.66.21.164 proto esp spi
0x301 mode transport aead "rfc4106(gcm(aes))"
0x01020304f6ddb555acfd9d77b03ea3843f265325 96
#
# Policy
if [ "${NODE}" = "A" ]; then
./ip xfrm policy add dir out src 10.66.21.164 dst 10.66.21.166
tmpl proto esp mode transport
./ip xfrm policy add dir in src 10.66.21.166 dst 10.66.21.164
tmpl proto esp mode transport
fi
if [ "${NODE}" = "B" ]; then
./ip xfrm policy add dir in src 10.66.21.164 dst 10.66.21.166
tmpl proto esp mode transport
./ip xfrm policy add dir out src 10.66.21.166 dst 10.66.21.164
tmpl proto esp mode transport
fi

2010年9月3日 星期五

PF_KEYv2 to cipher support internal(二)

這次,我從一個crypto_null.ko的crypto module開始
如同其名,這是一個不做任何事情的cihper
Kernel version 2.6.10

crypto/crypto_null.c
145 static int __init crypto_null_mod_init(void)
146 {
147         int ret = 0;
148
149         ret = crypto_register_alg(&cipher_null);
150         if (ret < 0)
151                 goto out;
152
153         ret = crypto_register_alg(&skcipher_null);
154         if (ret < 0)
155                 goto out_unregister_cipher;
156
157         ret = crypto_register_alg(&digest_null);
158         if (ret < 0)
159                 goto out_unregister_skcipher;
160
161         ret = crypto_register_alg(&compress_null);
162         if (ret < 0)
163                 goto out_unregister_digest
crypto_register_alg will add struct list_head crypto_register_alg.
107 static struct crypto_alg cipher_null = {
108         .cra_name               =       "cipher_null",
109         .cra_flags              =       CRYPTO_ALG_TYPE_CIPHER,
110         .cra_blocksize          =       NULL_BLOCK_SIZE,
111         .cra_ctxsize            =       0,
112         .cra_module             =       THIS_MODULE,
113         .cra_list               =       LIST_HEAD_INIT(cipher_null.cra_list),
114         .cra_u                  =       { .cipher = {
115         .cia_min_keysize        =       NULL_KEY_SIZE,
116         .cia_max_keysize        =       NULL_KEY_SIZE,
117         .cia_setkey             =       null_setkey,
118         .cia_encrypt            =       null_crypt,
119         .cia_decrypt            =       null_crypt } }
120 };
Both of cia_encrypt,cia_decrypt do nothing just call null_crypt

55 static void null_crypt(struct crypto_tfm *tfm, u8 *dst, const u8 *src)
 56 {
 57         memcpy(dst, src, NULL_BLOCK_SIZE);
 58 }
But in blockcipher is different
122 static struct crypto_alg skcipher_null = {
123         .cra_name               =       "ecb(cipher_null)",
124         .cra_driver_name        =       "ecb-cipher_null",
125         .cra_priority           =       100,
126         .cra_flags              =       CRYPTO_ALG_TYPE_BLKCIPHER,
127         .cra_blocksize          =       NULL_BLOCK_SIZE,
128         .cra_type               =       &crypto_blkcipher_type,
129         .cra_ctxsize            =       0,
130         .cra_module             =       THIS_MODULE,
131         .cra_list               =       LIST_HEAD_INIT(skcipher_null.cra_list),
132         .cra_u                  =       { .blkcipher = {
133         .min_keysize            =       NULL_KEY_SIZE,
134         .max_keysize            =       NULL_KEY_SIZE,
135         .ivsize                 =       NULL_IV_SIZE,
136         .setkey                 =       null_setkey,
137         .encrypt                =       skcipher_null_crypt,
138         .decrypt                =       skcipher_null_crypt } }
139 };

Blockcipher encrypt/decrypt有些不同其中,
(1).  首先inital一個struct blkcipher_walk 
        來記錄src 和dststruct scatterlist (一般這是和platform的dma有關)
            參考: http://lwn.net/Articles/263343/
                      http://lwn.net/Articles/256368/
(2). blkcipher_walk_virt用來

 60 static int skcipher_null_crypt(struct blkcipher_desc *desc,
 61                                struct scatterlist *dst,
 62                                struct scatterlist *src, unsigned int nbytes)
 63 {
 64         struct blkcipher_walk walk;
 65         int err;
 66
 67         blkcipher_walk_init(&walk, dst, src, nbytes);
 68         err = blkcipher_walk_virt(desc, &walk);
 69
 70         while (walk.nbytes) {
 71                 if (walk.src.virt.addr != walk.dst.virt.addr)
 72                         memcpy(walk.dst.virt.addr, walk.src.virt.addr,
 73                                walk.nbytes);
 74                 err = blkcipher_walk_done(desc, &walk, 0);
 75         }
 76
 77         return err;
 78 }
 79


blkcipher_walk describes the relationship of  physical / virtual address whare raw date stored in.
include/crypto/algapi.h
 65 struct scatter_walk {
 66         struct scatterlist *sg;
 67         unsigned int offset;
 68 };
The struct scatterlist is platform depend.
eg: arch/mips/include/asm/scatterlist.h
struct scatterlist {
#ifdef CONFIG_DEBUG_SG
        unsigned long   sg_magic;
#endif
        unsigned long   page_link;
        unsigned int    offset;
        dma_addr_t      dma_address;
        unsigned int    length;
};
 70 struct blkcipher_walk {
 71         union {
 72                 struct {
 73                         struct page *page;
 74                         unsigned long offset;
 75                 } phys;
 76
 77                 struct {
 78                         u8 *page;
 79                         u8 *addr;
 80                 } virt;
 81         } src, dst;
 82
 83         struct scatter_walk in;
 84         unsigned int nbytes;
 85
 86         struct scatter_walk out;
 87         unsigned int total;
 88
 89         void *page;
 90         u8 *buffer;
 91         u8 *iv;
 92
 93         int flags;
 94         unsigned int blocksize;
 95 };

再舉個(VIA PadLock hardware crypto engine)例子
drivers/crypto/padlock-aes.c
255 static int ecb_aes_encrypt(struct blkcipher_desc *desc,
256                            struct scatterlist *dst, struct scatterlist *src,
257                            unsigned int nbytes)
258 {
259         struct aes_ctx *ctx = blk_aes_ctx(desc->tfm);
260         struct blkcipher_walk walk;
261         int err;
262         int ts_state;
263
264         padlock_reset_key();
265
266         blkcipher_walk_init(&walk, dst, src, nbytes);
267         err = blkcipher_walk_virt(desc, &walk);
268
269         ts_state = irq_ts_save();
270         while ((nbytes = walk.nbytes)) {
271                 padlock_xcrypt_ecb(walk.src.virt.addr, walk.dst.virt.addr,
272                                    ctx->E, &ctx->cword.encrypt,
273                                    nbytes / AES_BLOCK_SIZE);
274                 nbytes &= AES_BLOCK_SIZE - 1;
275                 err = blkcipher_walk_done(desc, &walk, nbytes);
276         }
277         irq_ts_restore(ts_state);
278
279         return err;
280 }

再來看一下cipher_alg這個structure, 其中rca_u會根據不同的cipher:(eg: crypto/hash/digest)
在register時inital相對應的methods

linux/crypto.h
 322 struct crypto_alg {
 323         struct list_head cra_list;
 324         struct list_head cra_users;
 325
 326         u32 cra_flags;
 327         unsigned int cra_blocksize;
 328         unsigned int cra_ctxsize;
 329         unsigned int cra_alignmask;
 330
 331         int cra_priority;
 332         atomic_t cra_refcnt;
 333
 334         char cra_name[CRYPTO_MAX_ALG_NAME];
 335         char cra_driver_name[CRYPTO_MAX_ALG_NAME];
 336
 337         const struct crypto_type *cra_type;
 338
 339         union {
 340                 struct ablkcipher_alg ablkcipher;
 341                 struct aead_alg aead;
 342                 struct blkcipher_alg blkcipher;
 343                 struct cipher_alg cipher;
 344                 struct digest_alg digest;
 345                 struct hash_alg hash;
 346                 struct ahash_alg ahash;
 347                 struct compress_alg compress;
 348                 struct rng_alg rng;
 349         } cra_u;
 350
 351         int (*cra_init)(struct crypto_tfm *tfm);
 352         void (*cra_exit)(struct crypto_tfm *tfm);
 353         void (*cra_destroy)(struct crypto_alg *alg);
 354
 355         struct module *cra_module;
 356 };

2010年8月30日 星期一

trace netfilter conntrack nf_conntrack_hash


For some purpose, I need to reduce netfilter connect track time as 1 to expire ASAP.
First all, I need to know how to list /proc/net/ip_conntrack.

在2.6.x早期還是用nf_conntrack_hash為bucket的hash list,所以還看得到如下的global varibles

net/netfilter/nf_conntrack_core.c
struct hlist_head *nf_conntrack_hash __read_mostly;
EXPORT_SYMBOL_GPL(nf_conntrack_hash);
struct nf_conn nf_conntrack_untracked __read_mostly;
EXPORT_SYMBOL_GPL(nf_conntrack_untracked);
新的netfilter會將ipv4和ipv6結合起來, 且因為新的net_namespace架構始得netfilter的connect track有一些變化。
namespace的架構主要是因為linux kernel要support virtual machine,好處是security及virtualize
,但程式碼變得更難理解,(我看不太懂),且上面的這些原本是global varibles也hidden到net_namespace裡了…

anyway,I just want to resolve my problem in project: I need to reset connect expire time in traced sessions.(kernel 2.6.28x)
First, I trace the module: nf_conntrack_ipv4.ko, when module initical, it will create /proc/net/ip_conntrack and /proc/net/nf_conntrack
nf_conntrack_l3proto_ipv4_compat.c
static int __net_init ip_conntrack_net_init(struct net *net)
{
        struct proc_dir_entry *proc, *proc_exp, *proc_stat;

        proc = proc_net_fops_create(net, "ip_conntrack", 0440, &ct_file_ops);
        if (!proc)
                goto err1;

        proc_exp = proc_net_fops_create(net, "ip_conntrack_expect", 0440,
                                        &ip_exp_file_ops);
        if (!proc_exp)
                goto err2;

        proc_stat = proc_create("ip_conntrack", S_IRUGO,
                                net->proc_net_stat, &ct_cpu_seq_fops);


nf_conntrack_l3proto_ipv4_compat.c
static const struct file_operations ct_file_ops = {
        .owner   = THIS_MODULE,
        .open    = ct_open,
        .read    = seq_read,
        .llseek  = seq_lseek,
        .release = seq_release_net,
};


nf_conntrack_l3proto_ipv4_compat.c

static int ct_open(struct inode *inode, struct file *file)
{
        return seq_open_net(inode, file, &ct_seq_ops,
                            sizeof(struct ct_iter_state));//private data stored in seq_xxx
}


nf_conntrack_l3proto_ipv4_compat.c

static const struct seq_operations ct_seq_ops = {
        .start = ct_seq_start,
        .next  = ct_seq_next,
        .stop  = ct_seq_stop,
        .show  = ct_seq_show
};
fs/proc/proc_net.c
int seq_open_net(struct inode *ino, struct file *f,
                 const struct seq_operations *ops, int size)
{
        struct net *net;
        struct seq_net_private *p;

        BUG_ON(size < sizeof(*p));

        net = get_proc_net(ino);
        if (net == NULL)
                return -ENXIO;

        p = __seq_open_private(f, ops, size);
        if (p == NULL) {
                put_net(net);
                return -ENOMEM;
        }
#ifdef CONFIG_NET_NS
        p->net = net;
#endif
        return 0;
}
EXPORT_SYMBOL_GPL(seq_open_net);
Finally, the ct_seq_show dispaly each of session.

static int ct_seq_show(struct seq_file *s, void *v)
{
        const struct nf_conntrack_tuple_hash *hash = v;
        const struct nf_conn *ct = nf_ct_tuplehash_to_ctrack(hash);
        const struct nf_conntrack_l3proto *l3proto;
        const struct nf_conntrack_l4proto *l4proto;

        NF_CT_ASSERT(ct);

        /* we only want to print DIR_ORIGINAL */
        if (NF_CT_DIRECTION(hash))
                return 0;
        if (nf_ct_l3num(ct) != AF_INET)
                return 0;

        l3proto = __nf_ct_l3proto_find(nf_ct_l3num(ct));
        NF_CT_ASSERT(l3proto);
        l4proto = __nf_ct_l4proto_find(nf_ct_l3num(ct), nf_ct_protonum(ct));
        NF_CT_ASSERT(l4proto);
Now!, I try to modify expire time of each session.by mod_timer, eg:
 mod_timer(&ct->timeout, jiffies + HZ/2); /* update expire as 0.5sec. */
after recompiled mdoules, just install it
#insmod nf_conntrack_ipv4.ko
then read again to run mod_timer for each sessions
#cat /proc/net/ip_conntrack
Great!, all sessions  will be deleted after 0.5 sec


--------------------------------------------------------------------------

Extra topic from this case:

net/core/net_namespace.c
int register_pernet_subsys(struct pernet_operations *ops)
{
        int error;
        mutex_lock(&net_mutex);
        error =  register_pernet_operations(first_device, ops);
        mutex_unlock(&net_mutex);
        return error;
}
EXPORT_SYMBOL_GPL(register_pernet_subsys);

include/net/net_namespace.h
struct pernet_operations {
        struct list_head list;
        int (*init)(struct net *net);
        void (*exit)(struct net *net);
};
register_pernet_operations: will call method init from each of net_namespace which regiestered.
But, where or how to get hash bucket by net namesapce....?

-----------------------------------
Used functions in this case
/include/linux/moduleparam.h
#define module_param_call(name, set, get, arg, perm)                          \ __module_param_call(MODULE_PARAM_PREFIX, name, set, get, arg, perm)
rcu_dereference: include/linux/rcupdate.h
#define rcu_dereference(p)     ({ \
                                typeof(p) _________p1 = ACCESS_ONCE(p); \
                                smp_read_barrier_depends(); \
                                (_________p1); \
                                })
http://rd-life.blogspot.com/2009/05/rcu_26.html
http://lxr.linux.no/#linux+v2.6.28/Documentation/RCU/whatisRCU.txt#L122


2010年8月23日 星期一

PF_KEYv2 to cipher support internal

最近有個project因為kernel的cipher沒辦法support,所以要trace一下openswan和kernel modeules之前的operations

Frist one, I need to understand openswan how to communicate with kernel SA:
So I found it is through PF_KEYv2 socket family to interface key engine, see RFC2367

Then I found a open source sample from svn repository:
    http://xbq-code-repository.googlecode.com/svn/trunk/unpcode
This is come from sample code that famous "UNIX netorking programming" v1, 3rd.

The only thing you need to modify is the header file path is from net/pfkeyv2.h to linux/pfkeyv2.h
I only compile lib and libfree subdirecotry within unpcodoe.

You can use the 「register」program to get cipher support list from kernel throught PF_KEYv2 protocol.

/tmp/rootfs # ./register -t esp
Sending register message:
SADB Message Register, errno 0, satype IPsec ESP, seq 0, pid 17534

Reply returned:
SADB Message Register, errno 0, satype IPsec ESP, seq 0, pid 17534
 Supported authentication algorithms:
  Null ivlen 0 bits 0-0
  HMAC-MD5 ivlen 0 bits 128-128
  HMAC-SHA-1 ivlen 0 bits 160-160
 Supported encryption algorithms:
  Null ivlen 0 bits 0-0
  DES-CBC ivlen 8 bits 64-64
  3DES-CBC ivlen 8 bits 192-192
  [Unknown encryption algorithm 12] ivlen 8 bits 128-256
------------------------------------------------------------------------------------------

Now, I still can not get blowfish, twofish, etc.. cipher support from PF_KEYv2, even I get list form /proc/crypto

/tmp/rootfs # cat /proc/crypto |grep driver
driver       : authenc(hmac(sha1-ubicom32),cbc-aes-ubicom32)
....
driver       : michael_mic-generic
driver       : ecb(arc4-generic)
driver       : krng
driver       : seed-generic
driver       : arc4-generic
driver       : cast6-generic
driver       : cast5-generic
driver       : tnepres-generic
driver       : serpent-generic
driver       : twofish-generic
driver       : blowfish-generic
driver       : sha1-generic
...
So the next step is to get understand  the kernel layers of crypto, xfrm and af_keyv2.

net/key/af_key.c: An implamentation of AF_KEYv2
The AF_KEYv2 type 'REGISTER' will call

     static int pfkey_register(struct sock *sk, struct sk_buff *skb, struct sadb_msg *hdr, void **ext_hdrs)

then it call

     static struct sk_buff *compose_sadb_supported(struct sadb_msg *orig,
                                              gfp_t allocation)

Of course, the main task of  compose_sadb_supported is to compose auth and crypto supported list into skb.


The important subfounctions called by compose_sadb_supported are
  1. xfrm_count_auth_supported: to get the number of auth algs supported
  2. xfrm_count_enc_supported: to get the number of crypto algs supported
  3. if auth present, for each  xfrm_aalg_get_byidx
  4. if crypto present, for each  xfrm_ealg_get_byidx
another interesting functions is "void xfrm_probe_algs(void)"
It probes/init internal static lists of xfrm layer :aalg_list, ealg_list, calg_list,
by  crypto_has_hash(), crypto_has_blkcipher(), crypto_has_comp().
and the crypto_has_xxx() defined on include/linux/crypto.h eg:
static inline int crypto_has_blkcipher(const char *alg_name, u32 type, u32 mask)
{
        type &= ~CRYPTO_ALG_TYPE_MASK;
        type |= CRYPTO_ALG_TYPE_BLKCIPHER;
        mask |= CRYPTO_ALG_TYPE_MASK;

        return crypto_has_alg(alg_name, type, mask);
}
crypto_has_blkcipher() inital cipher mask for block cipher then look for by crypto API. Now let us trace them
crypto/api.c,
int crypto_has_alg(const char *name, u32 type, u32 mask)
{
        int ret = 0;
        struct crypto_alg *alg = crypto_alg_mod_lookup(name, type, mask);

        if (!IS_ERR(alg)) {
                crypto_mod_put(alg);
                ret = 1;
        }

        return ret;
}
struct crypto_alg *crypto_alg_mod_lookup(const char *name, u32 type, u32 mask)
{
        struct crypto_alg *alg;
        struct crypto_alg *larval;
        int ok;

        if (!(mask & CRYPTO_ALG_TESTED)) {
                type |= CRYPTO_ALG_TESTED;
                mask |= CRYPTO_ALG_TESTED;
        }

        larval = crypto_larval_lookup(name, type, mask);
        if (IS_ERR(larval) || !crypto_is_larval(larval))
                return larval;

        ok = crypto_probing_notify(CRYPTO_MSG_ALG_REQUEST, larval);

        if (ok == NOTIFY_STOP)
                alg = crypto_larval_wait(larval);
        else {
                crypto_mod_put(larval);
                alg = ERR_PTR(-ENOENT);
        }
        crypto_larval_kill(larval);
        return alg;
}
struct crypto_alg *crypto_larval_lookup(const char *name, u32 type, u32 mask)
{
        struct crypto_alg *alg;

        if (!name)
                return ERR_PTR(-ENOENT);

        mask &= ~(CRYPTO_ALG_LARVAL | CRYPTO_ALG_DEAD);
        type &= mask;

        alg = try_then_request_module(crypto_alg_lookup(name, type, mask),
                                      name);
        if (alg)
                return crypto_is_larval(alg) ? crypto_larval_wait(alg) : alg;

        return crypto_larval_add(name, type, mask);
}

int crypto_probing_notify(unsigned long val, void *v)
{
        int ok;

        ok = blocking_notifier_call_chain(&crypto_chain, val, v);
        if (ok == NOTIFY_DONE) {
                request_module("cryptomgr");
                ok = blocking_notifier_call_chain(&crypto_chain, val, v);
        }

        return ok;
}
crypto/internal.h
static inline int crypto_is_larval(struct crypto_alg *alg)
{
        return alg->cra_flags & CRYPTO_ALG_LARVAL;
}
    net/xfrm/xfrm_algo.c
    int xfrm_count_enc_supported(void)
    {
            int i, n;

            for (i = 0, n = 0; i < ealg_entries(); i++)
                    if (ealg_list[i].available)
                            n++;
            return n;
    }

    struct xfrm_algo_desc *xfrm_ealg_get_byidx(unsigned int idx)
    {
            if (idx >= ealg_entries())
                    return NULL;

            return &ealg_list[idx];
    }
    EXPORT_SYMBOL_GPL(xfrm_ealg_get_byidx);

    The static struct xfrm_algo_desc ealg_list and aalg_list are static array to
    describe algs of auth,crypto, compress, eg:
    net/xfrm/xfrm_algo.c
    static struct xfrm_algo_desc ealg_list[] = {
    {
            .name = "ecb(cipher_null)",
            .compat = "cipher_null",

            .uinfo = {
                    .encr = {
                            .blockbits = 8,
                            .defkeybits = 0,
                    }
            },

            .desc = {
                    .sadb_alg_id =  SADB_EALG_NULL,
                    .sadb_alg_ivlen = 0,
                    .sadb_alg_minbits = 0,
                    .sadb_alg_maxbits = 0
            }
    },
    {
            .name = "cbc(des)",
            .compat = "des",

            .uinfo = {
                    .encr = {
                            .blockbits = 64,
                            .defkeybits = 64,
                    }
            },

            .desc = {
                    .sadb_alg_id = SADB_EALG_DESCBC,
                    .sadb_alg_ivlen = 8,
                    .sadb_alg_minbits = 64,
                    .sadb_alg_maxbits = 64
            }
    },
    {
            .name = "cbc(des3_ede)",
            .compat = "des3_ede", .....
    ...

    --------------------------------------------------------------------------------

    For now, I change trace toward  from AF_KEYv2 to crpyto layer.
    For example, in NULL cipher common support
    crypto/crypto_null.c
    crypto_register_alg(&cipher_null)  called in module initial. 
    crypto/algapi.c:
    __crypto_register_alg was called finially, it will add to list by below if anything ok.
       o Check cipher is registered before for anything.
       o alloc "struct crypto_larval *larval"  by crypto_larval_alloc
       o Call  crypto_mod_get to requset module if need, and add a referance count,
          then retuen to larval->adult.
            larval->adult = crypto_mod_get(alg);
       o  Now anything is ready, add to list chain.

            list_add(&alg->cra_list, &crypto_alg_list);
            list_add(&larval->alg.cra_list, &crypto_alg_list);

    crpyto/api.c

    LIST_HEAD(crypto_alg_list);
    EXPORT_SYMBOL_GPL(crypto_alg_list);
    DECLARE_RWSEM(crypto_alg_sem);
    EXPORT_SYMBOL_GPL(crypto_alg_sem);

    BLOCKING_NOTIFIER_HEAD(crypto_chain);
    EXPORT_SYMBOL_GPL(crypto_chain);
    static inline struct crypto_alg *crypto_alg_get(struct crypto_alg *alg)
    {
            atomic_inc(&alg->cra_refcnt);
            return alg;
    }
    struct crypto_alg *crypto_mod_get(struct crypto_alg *alg)
    {
            return try_module_get(alg->cra_module) ? crypto_alg_get(alg) : NULL;
    }

    crpyto/internal.h

    struct crypto_larval {
            struct crypto_alg alg;
            struct crypto_alg *adult;
            struct completion completion;
            u32 mask;
    };
    It is interest to know about
    __crypto_register_alg
     待續