annotate vendor/plugins/coderay-0.9.2/lib/.svn/text-base/coderay.rb.svn-base @ 36:de76cd3e8c8e cc-branches

* Probably abortive experiments in extracting the branch from Hg
author Chris Cannam <chris.cannam@soundsoftware.ac.uk>
date Wed, 20 Oct 2010 10:07:29 +0100
parents 513646585e45
children
rev   line source
Chris@0 1 # = CodeRay Library
Chris@0 2 #
Chris@0 3 # CodeRay is a Ruby library for syntax highlighting.
Chris@0 4 #
Chris@0 5 # I try to make CodeRay easy to use and intuitive, but at the same time fully featured, complete,
Chris@0 6 # fast and efficient.
Chris@0 7 #
Chris@0 8 # See README.
Chris@0 9 #
Chris@0 10 # It consists mainly of
Chris@0 11 # * the main engine: CodeRay (Scanners::Scanner, Tokens/TokenStream, Encoders::Encoder), PluginHost
Chris@0 12 # * the scanners in CodeRay::Scanners
Chris@0 13 # * the encoders in CodeRay::Encoders
Chris@0 14 #
Chris@0 15 # Here's a fancy graphic to light up this gray docu:
Chris@0 16 #
Chris@0 17 # http://cycnus.de/raindark/coderay/scheme.png
Chris@0 18 #
Chris@0 19 # == Documentation
Chris@0 20 #
Chris@0 21 # See CodeRay, Encoders, Scanners, Tokens.
Chris@0 22 #
Chris@0 23 # == Usage
Chris@0 24 #
Chris@0 25 # Remember you need RubyGems to use CodeRay, unless you have it in your load path. Run Ruby with
Chris@0 26 # -rubygems option if required.
Chris@0 27 #
Chris@0 28 # === Highlight Ruby code in a string as html
Chris@0 29 #
Chris@0 30 # require 'coderay'
Chris@0 31 # print CodeRay.scan('puts "Hello, world!"', :ruby).html
Chris@0 32 #
Chris@0 33 # # prints something like this:
Chris@0 34 # puts <span class="s">&quot;Hello, world!&quot;</span>
Chris@0 35 #
Chris@0 36 #
Chris@0 37 # === Highlight C code from a file in a html div
Chris@0 38 #
Chris@0 39 # require 'coderay'
Chris@0 40 # print CodeRay.scan(File.read('ruby.h'), :c).div
Chris@0 41 # print CodeRay.scan_file('ruby.h').html.div
Chris@0 42 #
Chris@0 43 # You can include this div in your page. The used CSS styles can be printed with
Chris@0 44 #
Chris@0 45 # % coderay_stylesheet
Chris@0 46 #
Chris@0 47 # === Highlight without typing too much
Chris@0 48 #
Chris@0 49 # If you are one of the hasty (or lazy, or extremely curious) people, just run this file:
Chris@0 50 #
Chris@0 51 # % ruby -rubygems /path/to/coderay/coderay.rb > example.html
Chris@0 52 #
Chris@0 53 # and look at the file it created in your browser.
Chris@0 54 #
Chris@0 55 # = CodeRay Module
Chris@0 56 #
Chris@0 57 # The CodeRay module provides convenience methods for the engine.
Chris@0 58 #
Chris@0 59 # * The +lang+ and +format+ arguments select Scanner and Encoder to use. These are
Chris@0 60 # simply lower-case symbols, like <tt>:python</tt> or <tt>:html</tt>.
Chris@0 61 # * All methods take an optional hash as last parameter, +options+, that is send to
Chris@0 62 # the Encoder / Scanner.
Chris@0 63 # * Input and language are always sorted in this order: +code+, +lang+.
Chris@0 64 # (This is in alphabetical order, if you need a mnemonic ;)
Chris@0 65 #
Chris@0 66 # You should be able to highlight everything you want just using these methods;
Chris@0 67 # so there is no need to dive into CodeRay's deep class hierarchy.
Chris@0 68 #
Chris@0 69 # The examples in the demo directory demonstrate common cases using this interface.
Chris@0 70 #
Chris@0 71 # = Basic Access Ways
Chris@0 72 #
Chris@0 73 # Read this to get a general view what CodeRay provides.
Chris@0 74 #
Chris@0 75 # == Scanning
Chris@0 76 #
Chris@0 77 # Scanning means analysing an input string, splitting it up into Tokens.
Chris@0 78 # Each Token knows about what type it is: string, comment, class name, etc.
Chris@0 79 #
Chris@0 80 # Each +lang+ (language) has its own Scanner; for example, <tt>:ruby</tt> code is
Chris@0 81 # handled by CodeRay::Scanners::Ruby.
Chris@0 82 #
Chris@0 83 # CodeRay.scan:: Scan a string in a given language into Tokens.
Chris@0 84 # This is the most common method to use.
Chris@0 85 # CodeRay.scan_file:: Scan a file and guess the language using FileType.
Chris@0 86 #
Chris@0 87 # The Tokens object you get from these methods can encode itself; see Tokens.
Chris@0 88 #
Chris@0 89 # == Encoding
Chris@0 90 #
Chris@0 91 # Encoding means compiling Tokens into an output. This can be colored HTML or
Chris@0 92 # LaTeX, a textual statistic or just the number of non-whitespace tokens.
Chris@0 93 #
Chris@0 94 # Each Encoder provides output in a specific +format+, so you select Encoders via
Chris@0 95 # formats like <tt>:html</tt> or <tt>:statistic</tt>.
Chris@0 96 #
Chris@0 97 # CodeRay.encode:: Scan and encode a string in a given language.
Chris@0 98 # CodeRay.encode_tokens:: Encode the given tokens.
Chris@0 99 # CodeRay.encode_file:: Scan a file, guess the language using FileType and encode it.
Chris@0 100 #
Chris@0 101 # == Streaming
Chris@0 102 #
Chris@0 103 # Streaming saves RAM by running Scanner and Encoder in some sort of
Chris@0 104 # pipe mode; see TokenStream.
Chris@0 105 #
Chris@0 106 # CodeRay.scan_stream:: Scan in stream mode.
Chris@0 107 #
Chris@0 108 # == All-in-One Encoding
Chris@0 109 #
Chris@0 110 # CodeRay.encode:: Highlight a string with a given input and output format.
Chris@0 111 #
Chris@0 112 # == Instanciating
Chris@0 113 #
Chris@0 114 # You can use an Encoder instance to highlight multiple inputs. This way, the setup
Chris@0 115 # for this Encoder must only be done once.
Chris@0 116 #
Chris@0 117 # CodeRay.encoder:: Create an Encoder instance with format and options.
Chris@0 118 # CodeRay.scanner:: Create an Scanner instance for lang, with '' as default code.
Chris@0 119 #
Chris@0 120 # To make use of CodeRay.scanner, use CodeRay::Scanner::code=.
Chris@0 121 #
Chris@0 122 # The scanning methods provide more flexibility; we recommend to use these.
Chris@0 123 #
Chris@0 124 # == Reusing Scanners and Encoders
Chris@0 125 #
Chris@0 126 # If you want to re-use scanners and encoders (because that is faster), see
Chris@0 127 # CodeRay::Duo for the most convenient (and recommended) interface.
Chris@0 128 module CodeRay
Chris@0 129
Chris@0 130 $CODERAY_DEBUG ||= false
Chris@0 131
Chris@0 132 # Version: Major.Minor.Teeny[.Revision]
Chris@0 133 # Major: 0 for pre-stable, 1 for stable
Chris@0 134 # Minor: feature milestone
Chris@0 135 # Teeny: development state, 0 for pre-release
Chris@0 136 # Revision: Subversion Revision number (generated on rake gem:make)
Chris@0 137 VERSION = '0.9.2'
Chris@0 138
Chris@0 139 require 'coderay/tokens'
Chris@0 140 require 'coderay/token_classes'
Chris@0 141 require 'coderay/scanner'
Chris@0 142 require 'coderay/encoder'
Chris@0 143 require 'coderay/duo'
Chris@0 144 require 'coderay/style'
Chris@0 145
Chris@0 146
Chris@0 147 class << self
Chris@0 148
Chris@0 149 # Scans the given +code+ (a String) with the Scanner for +lang+.
Chris@0 150 #
Chris@0 151 # This is a simple way to use CodeRay. Example:
Chris@0 152 # require 'coderay'
Chris@0 153 # page = CodeRay.scan("puts 'Hello, world!'", :ruby).html
Chris@0 154 #
Chris@0 155 # See also demo/demo_simple.
Chris@0 156 def scan code, lang, options = {}, &block
Chris@0 157 scanner = Scanners[lang].new code, options, &block
Chris@0 158 scanner.tokenize
Chris@0 159 end
Chris@0 160
Chris@0 161 # Scans +filename+ (a path to a code file) with the Scanner for +lang+.
Chris@0 162 #
Chris@0 163 # If +lang+ is :auto or omitted, the CodeRay::FileType module is used to
Chris@0 164 # determine it. If it cannot find out what type it is, it uses
Chris@0 165 # CodeRay::Scanners::Plaintext.
Chris@0 166 #
Chris@0 167 # Calls CodeRay.scan.
Chris@0 168 #
Chris@0 169 # Example:
Chris@0 170 # require 'coderay'
Chris@0 171 # page = CodeRay.scan_file('some_c_code.c').html
Chris@0 172 def scan_file filename, lang = :auto, options = {}, &block
Chris@0 173 file = IO.read filename
Chris@0 174 if lang == :auto
Chris@0 175 require 'coderay/helpers/file_type'
Chris@0 176 lang = FileType.fetch filename, :plaintext, true
Chris@0 177 end
Chris@0 178 scan file, lang, options = {}, &block
Chris@0 179 end
Chris@0 180
Chris@0 181 # Scan the +code+ (a string) with the scanner for +lang+.
Chris@0 182 #
Chris@0 183 # Calls scan.
Chris@0 184 #
Chris@0 185 # See CodeRay.scan.
Chris@0 186 def scan_stream code, lang, options = {}, &block
Chris@0 187 options[:stream] = true
Chris@0 188 scan code, lang, options, &block
Chris@0 189 end
Chris@0 190
Chris@0 191 # Encode a string in Streaming mode.
Chris@0 192 #
Chris@0 193 # This starts scanning +code+ with the the Scanner for +lang+
Chris@0 194 # while encodes the output with the Encoder for +format+.
Chris@0 195 # +options+ will be passed to the Encoder.
Chris@0 196 #
Chris@0 197 # See CodeRay::Encoder.encode_stream
Chris@0 198 def encode_stream code, lang, format, options = {}
Chris@0 199 encoder(format, options).encode_stream code, lang, options
Chris@0 200 end
Chris@0 201
Chris@0 202 # Encode a string.
Chris@0 203 #
Chris@0 204 # This scans +code+ with the the Scanner for +lang+ and then
Chris@0 205 # encodes it with the Encoder for +format+.
Chris@0 206 # +options+ will be passed to the Encoder.
Chris@0 207 #
Chris@0 208 # See CodeRay::Encoder.encode
Chris@0 209 def encode code, lang, format, options = {}
Chris@0 210 encoder(format, options).encode code, lang, options
Chris@0 211 end
Chris@0 212
Chris@0 213 # Highlight a string into a HTML <div>.
Chris@0 214 #
Chris@0 215 # CSS styles use classes, so you have to include a stylesheet
Chris@0 216 # in your output.
Chris@0 217 #
Chris@0 218 # See encode.
Chris@0 219 def highlight code, lang, options = { :css => :class }, format = :div
Chris@0 220 encode code, lang, format, options
Chris@0 221 end
Chris@0 222
Chris@0 223 # Encode pre-scanned Tokens.
Chris@0 224 # Use this together with CodeRay.scan:
Chris@0 225 #
Chris@0 226 # require 'coderay'
Chris@0 227 #
Chris@0 228 # # Highlight a short Ruby code example in a HTML span
Chris@0 229 # tokens = CodeRay.scan '1 + 2', :ruby
Chris@0 230 # puts CodeRay.encode_tokens(tokens, :span)
Chris@0 231 #
Chris@0 232 def encode_tokens tokens, format, options = {}
Chris@0 233 encoder(format, options).encode_tokens tokens, options
Chris@0 234 end
Chris@0 235
Chris@0 236 # Encodes +filename+ (a path to a code file) with the Scanner for +lang+.
Chris@0 237 #
Chris@0 238 # See CodeRay.scan_file.
Chris@0 239 # Notice that the second argument is the output +format+, not the input language.
Chris@0 240 #
Chris@0 241 # Example:
Chris@0 242 # require 'coderay'
Chris@0 243 # page = CodeRay.encode_file 'some_c_code.c', :html
Chris@0 244 def encode_file filename, format, options = {}
Chris@0 245 tokens = scan_file filename, :auto, get_scanner_options(options)
Chris@0 246 encode_tokens tokens, format, options
Chris@0 247 end
Chris@0 248
Chris@0 249 # Highlight a file into a HTML <div>.
Chris@0 250 #
Chris@0 251 # CSS styles use classes, so you have to include a stylesheet
Chris@0 252 # in your output.
Chris@0 253 #
Chris@0 254 # See encode.
Chris@0 255 def highlight_file filename, options = { :css => :class }, format = :div
Chris@0 256 encode_file filename, format, options
Chris@0 257 end
Chris@0 258
Chris@0 259 # Finds the Encoder class for +format+ and creates an instance, passing
Chris@0 260 # +options+ to it.
Chris@0 261 #
Chris@0 262 # Example:
Chris@0 263 # require 'coderay'
Chris@0 264 #
Chris@0 265 # stats = CodeRay.encoder(:statistic)
Chris@0 266 # stats.encode("puts 17 + 4\n", :ruby)
Chris@0 267 #
Chris@0 268 # puts '%d out of %d tokens have the kind :integer.' % [
Chris@0 269 # stats.type_stats[:integer].count,
Chris@0 270 # stats.real_token_count
Chris@0 271 # ]
Chris@0 272 # #-> 2 out of 4 tokens have the kind :integer.
Chris@0 273 def encoder format, options = {}
Chris@0 274 Encoders[format].new options
Chris@0 275 end
Chris@0 276
Chris@0 277 # Finds the Scanner class for +lang+ and creates an instance, passing
Chris@0 278 # +options+ to it.
Chris@0 279 #
Chris@0 280 # See Scanner.new.
Chris@0 281 def scanner lang, options = {}
Chris@0 282 Scanners[lang].new '', options
Chris@0 283 end
Chris@0 284
Chris@0 285 # Extract the options for the scanner from the +options+ hash.
Chris@0 286 #
Chris@0 287 # Returns an empty Hash if <tt>:scanner_options</tt> is not set.
Chris@0 288 #
Chris@0 289 # This is used if a method like CodeRay.encode has to provide options
Chris@0 290 # for Encoder _and_ scanner.
Chris@0 291 def get_scanner_options options
Chris@0 292 options.fetch :scanner_options, {}
Chris@0 293 end
Chris@0 294
Chris@0 295 end
Chris@0 296
Chris@0 297 # This Exception is raised when you try to stream with something that is not
Chris@0 298 # capable of streaming.
Chris@0 299 class NotStreamableError < Exception
Chris@0 300 def initialize obj
Chris@0 301 @obj = obj
Chris@0 302 end
Chris@0 303
Chris@0 304 def to_s
Chris@0 305 '%s is not Streamable!' % @obj.class
Chris@0 306 end
Chris@0 307 end
Chris@0 308
Chris@0 309 # A dummy module that is included by subclasses of CodeRay::Scanner an CodeRay::Encoder
Chris@0 310 # to show that they are able to handle streams.
Chris@0 311 module Streamable
Chris@0 312 end
Chris@0 313
Chris@0 314 end
Chris@0 315
Chris@0 316 # Run a test script.
Chris@0 317 if $0 == __FILE__
Chris@0 318 $stderr.print 'Press key to print demo.'; gets
Chris@0 319 # Just use this file as an example of Ruby code.
Chris@0 320 code = File.read(__FILE__)[/module CodeRay.*/m]
Chris@0 321 print CodeRay.scan(code, :ruby).html
Chris@0 322 end