Saturday, June 20, 2015

Aporias

1.
Actually
He likes word games
   Diffident
In the way they circle about
   Starlings in flight
   Or Seraphim.

2.
With sweeping gesture
   Left to right
Hands hang with head
Under the pressing of the sun:
   Weight of doubt
   Or will.

3.
Against open air, tumultuous sea
   Turtle green
The division is nowhere more evident
Where sand meets froth
   Stark, blinding glare
Wind balmed
   Until night.

4.
Cleanliness,
   Next to godliness
The echo of countless schoolmarms
Chiding, chilling - without regret
   Yeah, rather,
   Motherhood.

Friday, June 19, 2015

What The World Needs More Of

The interview with these two kids - Chris and Camryn Singleton - is available on BBC, but I wanted to pull out this remarkable commentary in a related article:

"People are hurting in Charleston. But for the hundreds who packed into the gymnasium at the Goose Creek High School, it was also a reminder of the importance of love.

Sharonda Singleton coached the girls' athletics team here. As her photo rested on an easel on the polished floors in the vast sports hall, her friends and family paid tribute. Speaking for the first time since the deadly attack on the AME church where she worshipped, Sharonda's two children, Chris and Camryn, told me they forgive the man who killed her.

 "We already forgive him and there's nothing but love from our side of the family," Chris told me.

Many will find this incomprehensible. Charleston is often called the Holy City for the number of churches it is home to, and the role religion plays here. For some, like Chris and Camryn, unwavering faith is the only way to turn such a devastating loss into something positive."

This immediately brought to mind the sayings of Fr Zosima in Dostoevsky's Brothers Karamazov, which not to many years ago were the source of a kind of epiphany for me that in a sense reoriented by own thinking:

"Strive to love your neighbor actively and indefatigably. In as far as you advance in love you will grow surer of the reality of God and of the immort
ality of your soul. If you attain to perfect self-forgetfulness in the love of your neighbor, then you will believe without doubt, and no doubt can possibly enter your soul. This has been tried. This is certain.

Above all, avoid falsehood, every kind of falsehood, especially falseness to yourself. Watch over your own deceitfulness and look into it every hour, every minute. Avoid being scornful, both to others and to yourself. What seems to you bad within you will grow purer from the very fact of your observing it in yourself. Avoid fear, too, though fear is only the consequence of every sort of falsehood. Never be frightened at your own faint-heartedness in attaining love. Don't be frightened overmuch even at your evil actions. I am sorry I can say nothing more consoling to you, for love in action is a harsh and dreadful thing compared with love in dreams. Love in dreams is greedy for immediate action, rapidly performed and in the sight of all. Men will even give their lives if only the ordeal does not last long but is soon over, with all looking on and applauding as though on the stage. But active love is labor and fortitude, and for some people too, perhaps, a complete science. But I predict that just when you see with horror that in spite of all your efforts you are getting farther from your goal instead of nearer to it—at that very moment I predict that you will reach it and behold clearly the miraculous power of the Lord who has been all the time loving and mysteriously guiding you."

....


"At some thoughts one stands perplexed, especially at the sight of men's sin, and wonders whether one should use force or humble love. Always decide to use humble love. If you resolve on that once for all, you may subdue the whole world. Loving humility is marvelously strong, the strongest of all things, and there is nothing else like it." 

....

"“Remember particularly that you cannot be a judge of anyone. For no one can judge a criminal until he recognizes that he is just such a criminal as the man standing before him, and that he perhaps is more than all men to blame for that crime. When he understands that, he will be able to be a judge. Though that sounds absurd, it is true. If I had been righteous myself, perhaps there would have been no criminal standing before me. If you can take upon yourself the crime of the criminal your heart is judging, take it at once, suffer for him yourself, and let him go without reproach. And even if the law itself makes you his judge, act in the same spirit so far as possible, for he will go away and condemn himself more bitterly than you have done. If, after your kiss, he goes away untouched, mocking at you, do not let that be a stumbling-block to you. It shows his time has not yet come, but it will come in due course. And if it come not, no matter; if not he, then another in his place will understand and suffer, and judge and condemn himself, and the truth will be fulfilled. Believe that, believe it without doubt; for in that lies all the hope and faith of the saints.”


This time, Chris and Camryn have moved me beyond words by living this reality.

Addendum/edit: more of this humbling love on display

Sunday, September 21, 2014

3 film non-meme

Riffing off previous post - was discussing with my wife last evening what we thought the three best "recent" films we had seen were. Here's my list:

1) Jia Zhangke's A Touch of Sin.

Reason: this is a powerful, powerful film that explores the effects of radical individualism, and economic inequality and of the overturning of normal, local, rooted communities. Banned by the Chinese government, it is as much a critique of the values of neoliberalism globally as it is of the current Chinese economic experiment.

2) Alejandro González Iñárritu's Biutiful.

Reason: a moving exploration of responsibility and ethics in the face of poverty, hopelessness and impending death. What do we make of the human spirit and our obligations to each other - and our obligations in the face of The Other?  Javier Bardem was birthed for this role - fantastic acting.

3) Pavel Lungin's The Island.

Reason: who is guilty before whom and for what? Take a director of Jewish background, give him a story that is loosely inspired by a hagiography of the fool-for-Christ Feofil of the Kieven Caves, and cast a retired-rock-star-current-recluse (Pyotr Mamonov) as a Orthodox monastic in the far north of Russia, and I would have quite low expectations for the outcome. What Lungin produced is instead not only his best film but I think one of the best films of the last 20 years.

Tuesday, September 16, 2014

Top 10 Book Meme

What books have most impacted me? I picked books I have returned to over and over. Yes, I know this is solipsistic to publish, but its a fascinating thing to think through. I'm sure the list will not look right in a few months anyway. But here I go...

1 The Brothers Karamazov, Fyodor Dostoyevsky
Reason: the best book ever written. Duh.

2 Iob, LXX
Reason: bad things happen to good people, quite often.

3 I Am a Cat, Natsume Soseki
Reason: comedy is good for the soul. This is the funniest book I've ever read.

4 The Symposium, Plato
Reason: love. And I'm an only partially reconstructed platonist.

5 Demons, Fyodor Dostoyevsky
Reason: explains a big part of the 20th century. Makes 1984 look like crude propaganda.

6 Also Spracht Zarathustra, Friedrich Nietzsche
Reason: Nietszche saw the enormity of the modern project clearly.

7 Life of Moses, Gregory of Nyssa
Reason: before structuralism, post structuralism, semiotics, and deconstruction, there was Gregory of Nyssa. And apokatastasis.

8 For the Time Being, WH Auden
Reason: aside from the fact that Auden is the best English language poet, this is a deeply moving meditation on Christmas in the anglophone experience. Read it several times each winter.

9 The Divine Comedy, Dante Alighieri
Reason: great art meets allegory meets beauty. Honestly, stuck with only one book this might be it.

10 Faust, Johann Wolfgang von Goethe
Reason: we are all compromised to one degree or another.

Saturday, July 26, 2014

Silence

Silence. Sometimes sought after, but in reality almost certainly feared - the absence of not just sound but voice. Silence is often associated with divine encounter - the neptic tradition of the Philokalia comes to mind - but also and perhaps more accurately with abandonment, divine or otherwise. I recently read Shusaku Endo's Silence, a remarkable work, dwelling on the theme of abandonment in the context of the extirpation of Kakure Kirishitan communities in Tokagawa Japan. Many resilient families survived and eventually came out of hiding in the liberalization in the mid-19th century, but the persecutions were terrible. Their story is deeply moving (sufficiently so that over time I find myself drawn to devotion to the image of Maria-Kannon). Endo's novel was not without controversy but remains one of the great literary accomplishments of the 20th century.

In fact, the reason for this post is a kind of double entendre on silence: the relative silence in literate western circles with respect to Japanese literature of the past century. Over the last month, I realized that virtually no one I had spoken with had read a single Japanese novel. Yet, like Russia of the 19th century, Japan produced a concentration of great writers and great novelists in the last 20th century that is set apart: the forces of of profound national changes (and defeat) created the crucible of great art. That art carries the distinctive aesthetic sense of Japan - a kind of openness of form, but is necessarily the carrier of universal, humanistic themes.

Endo is a writer in the post war period - the so-called third generation, and in my view the last of the wave of great Japanese literature. Read him. But don't stop - perhaps don't start - there. The early 20th century work of Natsume Soseki are a product of the Meiji period. In my view, Soseki is not only a father of Japenese literature but one of the greatest figures of world literature taken as a whole - I am a Cat remains one of my very favorite novels. Two troubling post-war novels by Yukio Mishima merit attention - Confessions of a Mask and the Sailor Who Fell From Grace with the Sea, both I would characterize broadly as existential masterpieces. The topic of identity in the face of westernization is also a moving theme in Osamu Dazai's No Longer Human. I hardly mean this as a complete survey - something in any case I am not qualified to provide -just a pointer toward something broader and important.

My encounter with contemporary Japanese literature - albeit limited - has been less impactful (I want to like Haruki Murakami in the same way I want to like Victor Pelevin, but both make me think of the distorted echo of something far better). And again like Russia, it is difficult to know what to make of Japan today - where its future will lead, whether it will see a cultural resurgence or decline. It is certain that its roots are deep and I hope she finds a way to draw on them and to flourish.


Wednesday, July 23, 2014

Spark: A Discussion

A great presentation, worth watching in its entirety.

With apologies to my Hadoop friends but this is good for you too.

Monday, July 21, 2014

Exactly Wrong

I normally avoid anything that smacks of a competitive discussion on what I consider to be a space for personal reflection. So while I want to disclose the fact that I am not disinterested in the points I am making from a professional standpoint, my main interest is to frame some architecture points that I think are extremely important for the maturation and success of the Hadoop ecosystem.

A few weeks back, Mike Olson of Cloudera spoke at Spark Summit on how Spark relates to the future of Hadoop. The presentation can be found here:

http://youtu.be/8kcdwnbHnJo

In particular I want to draw attention to the statement made at 1:45 in the presentation that describes Spark as the "natural successor to MapReduce" - it becomes clear very quickly that what Olson is talking about is batch processing. This is fascinating as everyone I've talked to immediately points out one obvious thing: Spark isn't a general purpose batch processing framework - that is not its design center. The whole point of Spark is to enable fast data access and interactivity.
 
The guys that clearly "get" Spark - unsurprisingly - are DataBricks. In talking with Ion and company, it's clear they understand the use cases where Spark shines - data scientist driven data exploration and algorithmic development, machine learning, etc. - things that take advantage of the memory mapping capabilities and speed of the framework. And they have offered an online service that allows users to rapidly extract value from cloud friendly datasets, which is smart.

Cloudera's idea of pushing SQL, Pig and other frameworks on to Spark is actually a step backwards - it is a proposal to recreate all the problems of MapReduce 1: it fails to understand the power of refactoring resource management away from the compute model. Spark would have to reinvent and mature models for multi-tenancy, resource managemnet, scheduling, security, scaleout, etc that are frankly already there today for Hadoop 2 with YARN.

The announcement of an intent to lead an implementation of Hive on Spark got some attention. This was something that I looked at carefully with my colleagues almost 2 years ago, so I'd like to make a few observations on why we didn't take this path then.

The first was maturity, in terms of the Spark implementation, of Hive itself, and Shark. Candidly, we knew Hive itself worked at scale but needed significant enhancement and refactoring for both new features on the SQL front and to work at interactive speeds. And we wanted to do all this in a way that did not compromise Hive's ability to work at scale - for real big data problems. So we focused on the mainstream of Hive and the development of a Dryad like runtime for optimal execution of operators in physical plans for SQL in a way that meshed deeply with YARN. That model took the learnings of the database community and scale out big data solutions and built on them "from the inside out", so to speak.

Anyone who has been tracking Hadoop for, oh, the last 2-3 years will understand intuitively the right architectural approach needs to be based on YARN. What I mean is that the query execution must - at the query task level - be composed of tasks that are administered directly by YARN. This is absolutely critical for multi-workload systems (this is one reason why a bolt on MPP solution is a mistake for Hadoop - it is at best a tactical model while the system evolves).  This is why we are working with the community on Tez, a low level framework for enabling YARN native domain specific execution engines. For Hive-on-Tez, Hive is the engine and Tez provides the YARN level integration for resource negotiation and coorindation for DAG execution: a DAG of native operators analogous the the execution model found in the MPP world (when people compare Tez and Spark, they are fundamentally confused - Spark could be run on Tez for example for a much deeper integration with Hadoop 2 for example). This model allows the full range of use cases from interactive to massive batch to be administered in a deeply integrated, YARN native way.

Spark will undoubtedly mature into a great tool for what it is designed for: in memory, interactive scenarios - generally script driven - and likely grow to subsume new use cases we aren't anticipating today. It is, however, exactly the wrong choice for scale out big data batch processing in anything like the near term; worse still, returning to a monolithic general purpose compute framework for all Hadoop models would be a huge regression and is a disastrously bad idea.