莫名烦躁，书也读不进去的时候，干脆就坐在一旁自己发呆。又到了人生的十字路口，就想起了大学的时候。
大学的时候十分拮据，有时候一天只能吃一个Burrito，看着好的硬件只能眼馋发呆，房租常常上个月接不上下个月。没钱订杂志，也没钱买GPU。
但是大学的时候又很自由，可以自己写自己爱写的东西，成天也没有更多的人生追求。看到一段不为人知的故事，听到一个聪明的解法，就能高兴半天。周四早上买杯咖啡吃个Donut也可以在草坪上坐着看半天书。就连上课也都是好玩的东西。听教授讲各种故事，和大家聊天讨论。无趣的课全都逃掉，把集群上分配的时间全都用光光，还被教授笑眯眯地教训。
自己现在会的三脚猫功夫，动不动就提的two-phase commit，Byzantine failure，全都是那时候试出来的学会的。现在掌握的那些iOS的小技巧，哪有这些名字听起来酷，还有Paxos！
或许清醒的时候我都想不明白自己是有多想回学校整天读书，和土著人一起画画，没事用OpenCL写个比ffmpeg快20倍的Encoder啊。
I was not a fan of low-level parallelism exploitation. The level of parallelism that OpenMP tries is too easy to get wrong. On the other hand, I’d prefer dispatch_apply in any cases for the precise reason to disfavor OpenMP: it is restrictive enough to be only useful for embarrassing parallel problems.
The belief I subscribed to as stated above comes from Amdahl’s law; and in my simplified world, the more points you have dispatch / join, the more sequential execution portion you will end up with. Therefore, further limits what you can gain from parallel executions. In the same spirit, I’d prefer long-running processes with limited message-passing at any given time.
It’s not just talking points. In past versions of ccv, I never exploited thread-level parallelism to speed up ccv’s core functions (SIMD is not in the discussion, it is total awesomeness.) for the simple belief that as long as a given function can finish under a reasonable time, thread-level parallelism should be left for the upper layer of your application stack. When you are running ccv on iOS or Android, even though there are multiple cores, you probably don’t want ccv_icf_detect_objects to occupy all of them while executing. On the other hand, if you deploy ccv to a server environment, you probably want to pin process per core, and don’t have ccv to over-reaching other cores for parallelism (as hopping could be expensive).
At least that was what I believed until very recently.
Now looking back, the idea of that there exists a right level of parallelism is probably wrong. Until past three or four years, we have a very coarse parallelism levels (I am discounting SIMD again as it is for me a processor hack). We have processes, which can live in remote or local, but any communication between them are done explicitly via messaging. We have threads, which lives locally (or for most of threads), and probably no formal messaging channel; the communications are done implicitly via shared memory and synchronization. Most of the time, register, L1/L2 cache, and memory access latencies are the separated concerns and left for specific optimization. In that pretty coarse world, communications done with shared memory access, which assumes uniform latency, or with messaging, which holds to be expensive (high latency, and the cost of memory copy). The de-facto way to do parallelism in that world, as I previously said, is to go with embarrassing parallelism (avoid communication at all cost, and prefer high throughput / high latency alternatives over low throughput / low latency alternatives) and see how far you can get away with it.
But during the past few years, we have had a much finer-grain parallelism model, which specifies access latency with regards to different parallelism options. Even discounting the rising tide of GPGPU programming, we have much more CPU cores on a single machine (thread + shared memory) and the performance penalty of not aware of the non-uniform memory access pattern will be unforgivable.
I don’t have a good plan in mind about how to adapt to this as reality is still very messy. However, there are some interesting observations: 1). in heterogeneous computing environment, we are still playing the old game of balancing out communication cost with computation cost, but in very different ways for different platforms; 2). thus, it is unlikely one well-crafted kernel will perform universally well; include a set of tunable kernels, as the case in libatlas or fftw3 should be the common practice when delivering performance centric software; 3). because it is impossible to have the right level of parallelism, exploit parallelism structure at every level, and let tuners / schedulers to figure out how to adapt to a specific computing environment is the better bet.
It would be quite some fun to experiment how to exploit parallelism at thread-level for functions like ccv_icf_detect_objects / ccv_dpm_detect_objects without exhaust CPU resources on mobile devices with ccv in the future.
年底的时候，也想找个有高山悬崖大海的地方坐着边喝茶边写年终总结啊。所以坐在这里，其他人都在边喝酒边聊天，只有我打开笔记本开始劈劈啪啪敲起来。
年初心里打的小算盘，只有自己知道。但是还好完成了不多一些，要不望着这样的景色，就该羞愧地跳下去了。去年十二月底的时候，突然觉得如果只做重要的事情，而自己活得一塌糊涂总有些不对，才想起来，从二十一岁到现在，我是什么成色一点都没变。那个胖胖的自负的不知检点的毫无自制力的放纵的小男孩从来没死去过，他在这具基因不良的肉体里面活得可欢实了。
就算从来没想过长大，但是没有自制力也挺糟糕的，就跟那些从我的餐篮里偷薯条的小喜鹊似的，早早地肝硬化是说定的事。
那么，今年最大的进步，或许就是用这具基因不良的肉体和仅仅比平常人好一点的智力管住了他。
所以，其他的也就顺理成章了。
11700，是今年跑的英里数。
去了CVPR，和好多人整夜整夜的聊天。
发布了ccv的0.5。
还有Facebook的1234567。
终于能把每个月订的杂志和Email给看完了。
读完了7、8本书。
第一次跑完了马拉松，然后是50K。
管住了冬天的季节性抑郁。
把烤Fred’s练到了炉火纯青。
重新拾起了GPGPU Programming。
认识了一些好有趣的新朋友和导师。
三年前的时候，我最喜欢的一个教授跟我说，你或许现在觉得大学里要做的事情千头百绪，以为等到工作就好了；可是到工作之后，才发现每天要完成的事情更多，只能安慰自己说等到公司被收购了就好了；可是到公司被收购了之后，又会有更多的事情等着你去完成。从现在开始，需要你去完成的事只会越来越多，不过幸好你也会发现，原来再多的事情，都能处理得过来。
想到这些话和Grimshaw，突然觉得惭愧了起来，原来我也并没有做太多的事，瓶子里的那个小人，他没有多努力啊。就在这样的遗憾中，开始期待明年。
When I told Paul that I was way over-paid, I meant it. No matter how genetically-defected my physical is, or mere above average intelligence I have, everything I possessed is derived from public common. I need to do whatever I can to contribute back, and anything I got as extra, are the display of how serendipitous I am. There is no God in my world, but I am thankful before every meal.
总说要抽时间写身边的人，但是时间总在作对。或是太懒，或是写到一半就分了神，结果到现在，一篇也没有交出来。
今早父亲发邮件说舅公去世了，反而觉得意外，以至于再发邮件去确认的确是舅公。知道之后一整天都是恍恍惚惚的。
从小其实是挺闹腾不听话，所以也没少挨过打。92年姥爷去世是第一次见到死人，僵僵地就在那不动了。伸手过去抓，下午还握着暖暖的大手，变得冰凉。那时候极度恐慌，不是对死亡害怕，而是对未知恐惧。后来也见多了断胳膊断腿的事故和血，但是那双冷冰冰的手一直在记忆中，让人心慌。
可是舅公不一样，他是那种一直让我安心觉得不会死去的静止坐标。小时候父母嫌我太吵闹，就扔到舅公家学写字。舅公写得一手漂亮的柳体，我们都不临字帖，而是临舅公写的字。开始临三字经，后来又临增广、千字文。现在看来都是些无甚营养的糟粕，当时却临得、背得、崇拜得很起劲。而舅公那秀气的字体，也跟着印在了脑海里。一横一竖，一撇一捺，仍然能记得个四五八七六。
和他的字相比，舅公实在是长得太不相称了。又高又壮，一副大嗓门，说话直接了当。七八十岁了夏天还光着个膀子坐在院子里乘凉，露出一身的壮膘。别看我是一干这干那不安分的胖子，站在舅公身边，就是一弱不禁风的文弱书生。但是就是这么一个高壮的汉子，在舅婆生病起不来之后，细致地照顾了她五六年直到去世。或许两个人一起生活太久了，什么也都不算什么了，总之对于一直都一个人生活的我来说，复杂得难以想象。
父亲最后告诉我，舅公去世得很安然，因为肺癌来得很快，有症状已经是晚期，没受什么痛苦。可是我想，舅公也应该是安然地面对死亡的，因为七十年前，当他弟弟在战乱中无意义地被打死之后，是十几岁的他兀自找了一块地挖坑把弟弟的尸体埋上，然后继续上路的。
从92年到现在，二十一年了，原来我还是没办法面对死亡。
一开头就是竖Flag的节奏啊！虽然没人分享，不过生活真的很好！
能够读书，能够吃好吃的，能够周末去跑跑步，安定得难以形容。也只有这样才会能静下心来做些有意义的事情吧。
昨天早上突然开始在想Charles S. Peirce，想到他晚年的穷困潦倒。晚上睡觉的时候突然喘不过气，很担心自己心一紧，就这么无聊地死去。
美好的时光总有结束的时候吧，所以才觉得最近真需要抓紧时间做有意义的事，年底也不想回家了。
过了焦躁不安的春天和夏天之后，秋天来得格外安静和准时。我也变得特别安静和不想说话。本来在10月初的时候还准备写篇鸿文，结果拖来拖去换来了又一篇流水帐。大概因为调了三周CIFAR-10的参数，没调通之前都懒得和朋友们聊天和吃饭。还好每周末的跑步成了唯一自己和自己身体沟通的方式，还好还有唯一坚持的习惯。
很习惯这样呢。周中白天努力地工作，晚上调参。周末花大把的时间跑步和吃东西，还有调参。买许多的书不停地看，让自己思维没时间到处乱飞乱撞。就算尽量买Kindle版的，也积攒了不少的纸质书呢。我的「Think Higher, Feel Deeper」的书签却不知道扔哪找不到了！
十年前的那个秋天，我刚刚从家到上海，未来就像一张白纸从面前展开。
最近还在知乎上胡侃，大家一起来写吧！http://zhuanlan.zhihu.com/ultrarunning
上周六跑了二十英里之后被晒到差点昏死在Shoreline Park，结果今晚打球的时候已经凉到接二连三打喷嚏了。太阳公公啊，身材练好了之后还没时间去海边就到了九月，是不是辜负了整个夏天！
夏天过得这么快，就让人感叹人生。前两天和一哥们聊起人生迷惘，虽然说常拿此开玩笑，姑娘什么的都是再小不过的小事。脑子里整天想的，不过是为什么ICF训练要这么久怎么对齐convnet和调试CUDA为什么mosh不支持X11 Forward第二季度的经济复苏什么叫失业Fed的主席以及伊朗的核弹头和原来企业家都是有信仰这件事是真的为什么ultrarunner能够每周都有30多个小时的时间去跑步不能原谅。
现在才知道，Sigona’s的最后一批樱桃卖光的时候，夏天就已经结束了。爸爸生日快乐，再过四年，就一个甲子啦。
「喂，冲浪的时候，就是要把小腹的线条露出来给大家看的啊！」
最近疯狂地迷上了跑步。
刚开始是4英里，6英里，8英里。跑完Tough Mudder之后就想跑更多。跑完half marathon之后又想跑更远。于是10英里，14英里，20英里，24英里，中间又去跑了一个30k。虽然明明知道跑多远都没有意义，也知道Scott Jurek说的素食菜谱不符合营养学，可是大脑控制不住身体的反应，那些多巴胺把身体骗得一愣一愣的，每周六的早上六点钟，身体就从床上自己蹦下来。直到洗漱完毕，穿好衣服出门，大脑才清醒过来，嚷嚷道：「S**t, why did I sign up for this?!」
可是一上路，就只注意到自己的pace了，大脑开始嚷嚷着，「快了！快了！」或「慢了！慢了！」，试图让身体调节。到两英里的时候，是身体首先不干了，跟大脑抗议：「股四头肌好酸！」。右脑开始找理由：「大概是周四的jump squat，早就告诉你今天不要出来，拖到周日养好身体再来！」可是自己继续跑着，跑着跑着，身体就不说话了，忙着调节各个器官做好准备呢。下一次它抗议，要等到10英里的时候，再下一次，要到19英里的时候，然后是22英里的时候。
有说过我喜欢做一些自己也不太知道怎么回事的事情么？这事完全体现了那点。于是直到跑10英里磨破了脚，才知道应该带点创可贴。直到跑16英里没办法移动腿了，才知道应该带点吃的。直到跑到21英里体竭，才知道应该买点盐片和带水。
可是我是有多喜欢那些分泌的多巴胺带来的快感啊。做一件事情，能够投入多少，就看到另一个数字往上跳多少，开心极了。这大概就是Yaniv说的上瘾型人格，应该离大麻，神奇小蘑菇之类的远点。
或许没有这么直接，或许只是大脑和身体玩一个小游戏，想看看在身体完全崩溃的情况下，靠意志力能让它继续行多远。我也好想看看。
Futurama这种动画片，反而有一些特别Geeky的小感动。比如Philip Fry的爸爸告诉他：「If I’m hard on you, it’s only ‘cause I want you to grow up strong and resilient. Someday, you may face adversities so preposterous, I can’t even conceive of them.」 比如Fry 「moved the stars themselves to write her a love note in the sky.」
昨晚看完了Futurama的Cold Warriors那集之后，就躺在床上一直想父母对我的好。
想折纸，父亲就买了一堆的折纸书，带一大摞一大摞的A4纸回家，天天陪我叠各种小动物、飞机、机器人。想玩水，就有气垫船、快艇和潜水艇模型在后院的小水池里面试验，中心课题是不让它们进水。想要投影仪，母亲带了一堆的小电灯泡、透镜和反光镜，教我做了一个可以投影底片的小机器。想做飞行器，母亲又给了我一堆的马达和轻质材料，让我自己研究。自己搞了几周没有成功后，爸爸又给我买了一架动力滑翔机模型，它是真的可以飞的！父母带着我郊游的时候，他们总是能叫出各种花花草草的名字，可惜笨得我一个都没记住。和另一个朋友聊起来，说自己既不知道各种花花草草的名字，又不会搭帐篷，实在是看不到希望能够做一个称职的父亲。后来家里又有了一台显微镜，没事可以看好多有趣的东西，比如切片树叶，或者根茎，看起来光怪陆离的。也养过蝌蚪、小鸡仔和小乌龟，都没长大过。
一直就这么被父母宠着不知天高地厚，于是在十六岁离家出走的时候，已经会使用电烙铁、补鞋订扣子熨衬衫，还有写代码。之后还仗着被父母宠着，做了好多不靠谱的事情，哪怕到十七岁十八岁了，还给自己买各种昂贵的玩具。过年的时候买个步行机器人，过生日的时候订做了一个Mars Rover的仿品。
奇奇怪怪地活到现在，总觉得自己什么都懂也什么都会做，其实那是因为被两个人溺爱着，看不到世界的尖酸刻薄罢了。
People know that I talked about this two for quite some time. Since I won’t work on either of them commercially (the self-driving vehicle is more like a hobby project than anything else), it is probably better to write it down than talk it repeatedly to friends until made myself boring.
Self-driving
“Top Gear” is a pretty entertaining show until it went offensive. In a recent episode, James May went to Nevada and tried to compete with an autonomous vehicle. Jeremy Clarkson apparently didn’t understand self-driving vehicle and claimed that was no use for day to day life: because you were in it, when it transports you. But then, in London, you have a real issue when comes parking your classic luxury cars.
Self-driving vehicle would be the affordable private driver for everyone. Take a step back and think about parking. A car with a private driver would never worry about parking. It drops you off, and it is someone else’s (driver’s) problem. For self-driving vehicles, the driver is the computer. It can drop you off at the exact location you want to be, and then park itself a few miles away. When you need it to pick you up, send a message and it will be there in minutes.
When you start to consider the machine from utilitarian perspective, there are two contradicting thinkings. On one hand, the ownership over the vehicle would not be as important as of now. It is a machine that brings you from one place to another, and that’s it. Why own it if you can use it wherever you are, whenever you want? On the other hand, it could be extremely personal, because a vehicle is not for a whole family any more. It is for you, and yours only. It carries you around, like you carry your smartphone around. You don’t need to drop your kids to school, and then your wife to work. The vehicle carries them around, on their schedule.
In a self-driving vehicle world, these vehicles can get awkwardly small, think about SMART. There are some interesting aspects to urban planning. But other than we will get rid of all parking structures in the city, I haven’t explored much (or have any experience in urban planning).
Glass
When people think about Glass, now it always involves privacy. Privacy is a solvable problem. Companies or unions can mandate a chip into Glass and then you have devices that can disable Glass in certain areas (like restroom, poker room etc.). Putting that aside, at its basic, Glass is a neat and affordable head-mounted display (HMD). We were under the impression that Glass should be a hand-free device, and cornered ourselves in the imperfect world of speech recognition. Talking to HMD is a very awkward situation. Even Bluetooth headphone is there for a good 5-year, the world is still not ready for out-of-place conversation between man and air.
Glass is intriguing because it is the first commercialized everyday head-mounted display. You can take a casual 10-mile run with it, and Glass will tell you how fast you did, how far you’ve covered, and where is the next turn, you can even compete with your previous self, in real-time.
When is the last time you ordered coffee? What if a barista can remember your preference perfectly and smoothly because Glass recognizes you from a month ago?
When dreaming up these applications, we usually are taken to a futuristic direction far, and naturally, stretching Glass from a display interface to a new interaction interface. Misled by Javis in Ironman, we want it to listen to us, and that seems to be the only reasonable way to interact with such a futuristic device. It doesn’t have to be. There is nothing wrong with a small pocket size physical keyboard, as long as it works reliably. Is that not “natural” any more (“natural” in the same sense of now-dominate touch-screen interface)? A touch-screen interface creates a dynamic surface that gets mapped on to physical objects. A head-mounted display creates a dynamic surface in a fixed position of your view. There won’t be a direct interaction because it doesn’t map to a physical object.
Transformative Technologies
There are three kinds of transformative technologies. 1). The technical transformative one: things like smartphone, LED, or color film. Once these things are invented, it almost takes no time to get adopted and wide-spread. 2). The cultural transformative one: things like electric vehicle, TV or mobile phone. It replaces old cultural symbols (petrol station, news distribution, or public telephone), sometimes even delivers temporary social shocks (people can find you 24x7!), but given 10 to 20 years, its superiority will convince majority. 3). The institutional transformative one: things like online education, printing machine, or spinning jenny. These transformative technologies are most disruptive ones, they rendered the old institutions that we set up obsolete. It takes a good long time struggling, and the best people of our time, to make these institutional changes, I have nothing but only admiration to them.