I was not a fan of low-level parallelism exploitation. The level of parallelism that OpenMP tries is too easy to get wrong. On the other hand, I'd prefer dispatch_apply in any cases for the precise reason to disfavor OpenMP: it is restrictive enough to be only useful for embarrassing parallel problems.
The belief I subscribed to as stated above comes from Amdahl's law; and in my simplified world, the more points you have dispatch / join, the more sequential execution portion you will end up with. Therefore, further limits what you can gain from parallel executions. In the same spirit, I'd prefer long-running processes with limited message-passing at any given time.
It's not just talking points. In past versions of ccv, I never exploited thread-level parallelism to speed up ccv's core functions (SIMD is not in the discussion, it is total awesomeness.) for the simple belief that as long as a given function can finish under a reasonable time, thread-level parallelism should be left for the upper layer of your application stack. When you are running ccv on iOS or Android, even though there are multiple cores, you probably don't want ccv_icf_detect_objects to occupy all of them while executing. On the other hand, if you deploy ccv to a server environment, you probably want to pin process per core, and don't have ccv to over-reaching other cores for parallelism (as hopping could be expensive).
At least that was what I believed until very recently.
Now looking back, the idea of that there exists a right level of parallelism is probably wrong. Until past three or four years, we have a very coarse parallelism levels (I am discounting SIMD again as it is for me a processor hack). We have processes, which can live in remote or local, but any communication between them are done explicitly via messaging. We have threads, which lives locally (or for most of threads), and probably no formal messaging channel; the communications are done implicitly via shared memory and synchronization. Most of the time, register, L1/L2 cache, and memory access latencies are the separated concerns and left for specific optimization. In that pretty coarse world, communications done with shared memory access, which assumes uniform latency, or with messaging, which holds to be expensive (high latency, and the cost of memory copy). The de-facto way to do parallelism in that world, as I previously said, is to go with embarrassing parallelism (avoid communication at all cost, and prefer high throughput / high latency alternatives over low throughput / low latency alternatives) and see how far you can get away with it.
But during the past few years, we have had a much finer-grain parallelism model, which specifies access latency with regards to different parallelism options. Even discounting the rising tide of GPGPU programming, we have much more CPU cores on a single machine (thread + shared memory) and the performance penalty of not aware of the non-uniform memory access pattern will be unforgivable.
I don't have a good plan in mind about how to adapt to this as reality is still very messy. However, there are some interesting observations: 1). in heterogeneous computing environment, we are still playing the old game of balancing out communication cost with computation cost, but in very different ways for different platforms; 2). thus, it is unlikely one well-crafted kernel will perform universally well; include a set of tunable kernels, as the case in libatlas or fftw3 should be the common practice when delivering performance centric software; 3). because it is impossible to have the right level of parallelism, exploit parallelism structure at every level, and let tuners / schedulers to figure out how to adapt to a specific computing environment is the better bet.
It would be quite some fun to experiment how to exploit parallelism at thread-level for functions like ccv_icf_detect_objects / ccv_dpm_detect_objects without exhaust CPU resources on mobile devices with ccv in the future.
When I told Paul that I was way over-paid, I meant it. No matter how genetically-defected my physical is, or mere above average intelligence I have, everything I possessed is derived from public common. I need to do whatever I can to contribute back, and anything I got as extra, are the display of how serendipitous I am. There is no God in my world, but I am thankful before every meal.
昨天早上突然开始在想Charles S. Peirce，想到他晚年的穷困潦倒。晚上睡觉的时候突然喘不过气，很担心自己心一紧，就这么无聊地死去。
很习惯这样呢。周中白天努力地工作，晚上调参。周末花大把的时间跑步和吃东西，还有调参。买许多的书不停地看，让自己思维没时间到处乱飞乱撞。就算尽量买Kindle版的，也积攒了不少的纸质书呢。我的「Think Higher, Feel Deeper」的书签却不知道扔哪找不到了！
刚开始是4英里，6英里，8英里。跑完Tough Mudder之后就想跑更多。跑完half marathon之后又想跑更远。于是10英里，14英里，20英里，24英里，中间又去跑了一个30k。虽然明明知道跑多远都没有意义，也知道Scott Jurek说的素食菜谱不符合营养学，可是大脑控制不住身体的反应，那些多巴胺把身体骗得一愣一愣的，每周六的早上六点钟，身体就从床上自己蹦下来。直到洗漱完毕，穿好衣服出门，大脑才清醒过来，嚷嚷道：「S**t, why did I sign up for this?!」
Futurama这种动画片，反而有一些特别Geeky的小感动。比如Philip Fry的爸爸告诉他：「If I'm hard on you, it's only 'cause I want you to grow up strong and resilient. Someday, you may face adversities so preposterous, I can't even conceive of them.」 比如Fry 「moved the stars themselves to write her a love note in the sky.」
People know that I talked about this two for quite some time. Since I won't work on either of them commercially (the self-driving vehicle is more like a hobby project than anything else), it is probably better to write it down than talk it repeatedly to friends until made myself boring.
"Top Gear" is a pretty entertaining show until it went offensive. In a recent episode, James May went to Nevada and tried to compete with an autonomous vehicle. Jeremy Clarkson apparently didn't understand self-driving vehicle and claimed that was no use for day to day life: because you were in it, when it transports you. But then, in London, you have a real issue when comes parking your classic luxury cars.
Self-driving vehicle would be the affordable private driver for everyone. Take a step back and think about parking. A car with a private driver would never worry about parking. It drops you off, and it is someone else's (driver's) problem. For self-driving vehicles, the driver is the computer. It can drop you off at the exact location you want to be, and then park itself a few miles away. When you need it to pick you up, send a message and it will be there in minutes.
When you start to consider the machine from utilitarian perspective, there are two contradicting thinkings. On one hand, the ownership over the vehicle would not be as important as of now. It is a machine that brings you from one place to another, and that's it. Why own it if you can use it wherever you are, whenever you want? On the other hand, it could be extremely personal, because a vehicle is not for a whole family any more. It is for you, and yours only. It carries you around, like you carry your smartphone around. You don't need to drop your kids to school, and then your wife to work. The vehicle carries them around, on their schedule.
In a self-driving vehicle world, these vehicles can get awkwardly small, think about SMART. There are some interesting aspects to urban planning. But other than we will get rid of all parking structures in the city, I haven't explored much (or have any experience in urban planning).
When people think about Glass, now it always involves privacy. Privacy is a solvable problem. Companies or unions can mandate a chip into Glass and then you have devices that can disable Glass in certain areas (like restroom, poker room etc.). Putting that aside, at its basic, Glass is a neat and affordable head-mounted display (HMD). We were under the impression that Glass should be a hand-free device, and cornered ourselves in the imperfect world of speech recognition. Talking to HMD is a very awkward situation. Even Bluetooth headphone is there for a good 5-year, the world is still not ready for out-of-place conversation between man and air.
Glass is intriguing because it is the first commercialized everyday head-mounted display. You can take a casual 10-mile run with it, and Glass will tell you how fast you did, how far you've covered, and where is the next turn, you can even compete with your previous self, in real-time.
When is the last time you ordered coffee? What if a barista can remember your preference perfectly and smoothly because Glass recognizes you from a month ago?
When dreaming up these applications, we usually are taken to a futuristic direction far, and naturally, stretching Glass from a display interface to a new interaction interface. Misled by Javis in Ironman, we want it to listen to us, and that seems to be the only reasonable way to interact with such a futuristic device. It doesn't have to be. There is nothing wrong with a small pocket size physical keyboard, as long as it works reliably. Is that not "natural" any more ("natural" in the same sense of now-dominate touch-screen interface)? A touch-screen interface creates a dynamic surface that gets mapped on to physical objects. A head-mounted display creates a dynamic surface in a fixed position of your view. There won't be a direct interaction because it doesn't map to a physical object.
There are three kinds of transformative technologies. 1). The technical transformative one: things like smartphone, LED, or color film. Once these things are invented, it almost takes no time to get adopted and wide-spread. 2). The cultural transformative one: things like electric vehicle, TV or mobile phone. It replaces old cultural symbols (petrol station, news distribution, or public telephone), sometimes even delivers temporary social shocks (people can find you 24x7!), but given 10 to 20 years, its superiority will convince majority. 3). The institutional transformative one: things like online education, printing machine, or spinning jenny. These transformative technologies are most disruptive ones, they rendered the old institutions that we set up obsolete. It takes a good long time struggling, and the best people of our time, to make these institutional changes, I have nothing but only admiration to them.