Documentation for wukong lives at http://mrflip.github.com/wukong and in the repo itself, within the gh-pages branch.

If you’d like to contribute documentation here, please ping me and I’ll integrate it with the gh-pages; or (easier yet) just fork the repo and edit the gh-pages branch.

Go To the Hadoop Documentation

Index (possibly out of date)

• Install and set up wukong
  • Get the code
  • Setup
  • Installing and Running Wukong with Hadoop
  • Installing and Running Wukong with Datamapper, ActiveRecord, the command-line and more

• Tutorial
  • Count Words
  • Structured data
  • Accumulators including a UniqByLastReducer and a GroupBy reducer.

• Usage notes
  • How to run a Wukong script
  • How to test your scripts
  • Wukong Plays nicely with others
  • Schema export to Pig and SQL
  • Using wukong with internal streaming
  • Using wukong to Batch-Process ActiveRecord Objects

• Wutils — command-line utilies for working with data from the command line
  • Overview of wutils — command listing
  • Stupid command-line tricks using the wutils
  • wu-lign — present a tab-separated file as aligned columns
  • Dear Lazyweb, please build us a tab-oriented version of the Textutils library

• Links and tips for configuring and working with hadoop
• Some opinionated thoughts on working with big data, on why you should drop acid, treat exceptions as records, and happily embrace variable-length strings as primary keys.
• Wukong is licensed under the Apache License (same as Hadoop)

• More info
  • Why is it called Wukong?
  • Don’t Use Wukong, use this instead
  • Further Reading and useful links
  • Note on Patches/Pull Requests
  • What’s up with Wukong::AndPig?
  • Map/Reduce Algorithms
  • TODOs

• Work in progress: an intro to data processing with wukong:
  • Part 1, Get Ready
  • Part 2, Thinking Big Data
  • Part 3, Parsing