Adding new gem versions
Add the new gems referenced in the last commit for real.unstable
Tod Beardsley
parent
63e41ee6bb
commit
505b97b470
|
|
@ -0,0 +1,123 @@
|
|||
= CodeRay
|
||||
|
||||
Tired of blue'n'gray? Try the original version of this documentation on
|
||||
coderay.rubychan.de[http://coderay.rubychan.de/doc/] :-)
|
||||
|
||||
== About
|
||||
|
||||
CodeRay is a Ruby library for syntax highlighting.
|
||||
|
||||
You put your code in, and you get it back colored; Keywords, strings,
|
||||
floats, comments - all in different colors. And with line numbers.
|
||||
|
||||
*Syntax* *Highlighting*...
|
||||
* makes code easier to read and maintain
|
||||
* lets you detect syntax errors faster
|
||||
* helps you to understand the syntax of a language
|
||||
* looks nice
|
||||
* is what everybody wants to have on their website
|
||||
* solves all your problems and makes the girls run after you
|
||||
|
||||
|
||||
== Installation
|
||||
|
||||
% gem install coderay
|
||||
|
||||
|
||||
=== Dependencies
|
||||
|
||||
CodeRay needs Ruby 1.8.7+ or 1.9.2+. It also runs on Rubinius and JRuby.
|
||||
|
||||
|
||||
== Example Usage
|
||||
|
||||
require 'coderay'
|
||||
|
||||
html = CodeRay.scan("puts 'Hello, world!'", :ruby).div(:line_numbers => :table)
|
||||
|
||||
|
||||
== Documentation
|
||||
|
||||
See CodeRay.
|
||||
|
||||
|
||||
== Credits
|
||||
|
||||
=== Special Thanks to
|
||||
|
||||
* licenser (Heinz N. Gies) for ending my QBasic career, inventing the Coder
|
||||
project and the input/output plugin system.
|
||||
CodeRay would not exist without him.
|
||||
* bovi (Daniel Bovensiepen) for helping me out on various occasions.
|
||||
|
||||
=== Thanks to
|
||||
|
||||
* Caleb Clausen for writing RubyLexer (see
|
||||
http://rubyforge.org/projects/rubylexer) and lots of very interesting mail
|
||||
traffic
|
||||
* birkenfeld (Georg Brandl) and mitsuhiku (Arnim Ronacher) for PyKleur, now pygments.
|
||||
You guys rock!
|
||||
* Jamis Buck for writing Syntax (see http://rubyforge.org/projects/syntax)
|
||||
I got some useful ideas from it.
|
||||
* Doug Kearns and everyone else who worked on ruby.vim - it not only helped me
|
||||
coding CodeRay, but also gave me a wonderful target to reach for the Ruby
|
||||
scanner.
|
||||
* everyone who uses CodeBB on http://www.rubyforen.de and http://www.python-forum.de
|
||||
* iGEL, magichisoka, manveru, WoNáDo and everyone I forgot from rubyforen.de
|
||||
* Dethix from ruby-mine.de
|
||||
* zickzackw
|
||||
* Dookie (who is no longer with us...) and Leonidas from http://www.python-forum.de
|
||||
* Andreas Schwarz for finding out that CaseIgnoringWordList was not case
|
||||
ignoring! Such things really make you write tests.
|
||||
* closure for the first version of the Scheme scanner.
|
||||
* Stefan Walk for the first version of the JavaScript and PHP scanners.
|
||||
* Josh Goebel for another version of the JavaScript scanner, a SQL and a Diff scanner.
|
||||
* Jonathan Younger for pointing out the licence confusion caused by wrong LICENSE file.
|
||||
* Jeremy Hinegardner for finding the shebang-on-empty-file bug in FileType.
|
||||
* Charles Oliver Nutter and Yehuda Katz for helping me benchmark CodeRay on JRuby.
|
||||
* Andreas Neuhaus for pointing out a markup bug in coderay/for_redcloth.
|
||||
* 0xf30fc7 for the FileType patch concerning Delphi file extensions.
|
||||
* The folks at redmine.org - thank you for using and fixing CodeRay!
|
||||
* Keith Pitt for his SQL scanners
|
||||
* Rob Aldred for the terminal encoder
|
||||
* Trans for pointing out $DEBUG dependencies
|
||||
* Flameeyes for finding that Term::ANSIColor was obsolete
|
||||
* matz and all Ruby gods and gurus
|
||||
* The inventors of: the computer, the internet, the true color display, HTML &
|
||||
CSS, VIM, Ruby, pizza, microwaves, guitars, scouting, programming, anime,
|
||||
manga, coke and green ice tea.
|
||||
|
||||
Where would we be without all those people?
|
||||
|
||||
=== Created using
|
||||
|
||||
* Ruby[http://ruby-lang.org/]
|
||||
* Chihiro (my Sony VAIO laptop); Henrietta (my old MacBook);
|
||||
Triella, born Rico (my new MacBook); as well as
|
||||
Seras and Hikari (my PCs)
|
||||
* RDE[http://homepage2.nifty.com/sakazuki/rde_e.html],
|
||||
VIM[http://vim.org] and TextMate[http://macromates.com]
|
||||
* Subversion[http://subversion.tigris.org/]
|
||||
* Redmine[http://redmine.org/]
|
||||
* Firefox[http://www.mozilla.org/products/firefox/],
|
||||
Firebug[http://getfirebug.com/], Safari[http://www.apple.com/safari/], and
|
||||
Thunderbird[http://www.mozilla.org/products/thunderbird/]
|
||||
* RubyGems[http://docs.rubygems.org/] and Rake[http://rake.rubyforge.org/]
|
||||
* TortoiseSVN[http://tortoisesvn.tigris.org/] using Apache via
|
||||
XAMPP[http://www.apachefriends.org/en/xampp.html]
|
||||
* RDoc (though I'm quite unsatisfied with it)
|
||||
* Microsoft Windows (yes, I confess!) and MacOS X
|
||||
* GNUWin32, MinGW and some other tools to make the shell under windows a bit
|
||||
less useless
|
||||
* Term::ANSIColor[http://term-ansicolor.rubyforge.org/]
|
||||
* PLEAC[http://pleac.sourceforge.net/] code examples
|
||||
* Github
|
||||
* Travis CI (http://travis-ci.org/rubychan/github)
|
||||
|
||||
=== Free
|
||||
|
||||
* As you can see, CodeRay was created under heavy use of *free* software.
|
||||
* So CodeRay is also *free*.
|
||||
* If you use CodeRay to create software, think about making this software
|
||||
*free*, too.
|
||||
* Thanks :)
|
||||
|
|
@ -0,0 +1,35 @@
|
|||
$:.unshift File.dirname(__FILE__) unless $:.include? '.'
|
||||
|
||||
ROOT = '.'
|
||||
LIB_ROOT = File.join ROOT, 'lib'
|
||||
|
||||
task :default => :test
|
||||
|
||||
if File.directory? 'rake_tasks'
|
||||
|
||||
# load rake tasks from subfolder
|
||||
for task_file in Dir['rake_tasks/*.rake'].sort
|
||||
load task_file
|
||||
end
|
||||
|
||||
else
|
||||
|
||||
# fallback tasks when rake_tasks folder is not present (eg. in the distribution package)
|
||||
desc 'Run CodeRay tests (basic)'
|
||||
task :test do
|
||||
ruby './test/functional/suite.rb'
|
||||
ruby './test/functional/for_redcloth.rb'
|
||||
end
|
||||
|
||||
gem 'rdoc' if defined? gem
|
||||
require 'rdoc/task'
|
||||
desc 'Generate documentation for CodeRay'
|
||||
Rake::RDocTask.new :doc do |rd|
|
||||
rd.title = 'CodeRay Documentation'
|
||||
rd.main = 'README_INDEX.rdoc'
|
||||
rd.rdoc_files.add Dir['lib']
|
||||
rd.rdoc_files.add rd.main
|
||||
rd.rdoc_dir = 'doc'
|
||||
end
|
||||
|
||||
end
|
||||
|
|
@ -0,0 +1,215 @@
|
|||
#!/usr/bin/env ruby
|
||||
require 'coderay'
|
||||
|
||||
$options, args = ARGV.partition { |arg| arg[/^-[hv]$|--\w+/] }
|
||||
subcommand = args.first if /^\w/ === args.first
|
||||
subcommand = nil if subcommand && File.exist?(subcommand)
|
||||
args.delete subcommand
|
||||
|
||||
def option? *options
|
||||
!($options & options).empty?
|
||||
end
|
||||
|
||||
def tty?
|
||||
$stdout.tty? || option?('--tty')
|
||||
end
|
||||
|
||||
def version
|
||||
puts <<-USAGE
|
||||
CodeRay #{CodeRay::VERSION}
|
||||
USAGE
|
||||
end
|
||||
|
||||
def help
|
||||
puts <<-HELP
|
||||
This is CodeRay #{CodeRay::VERSION}, a syntax highlighting tool for selected languages.
|
||||
|
||||
usage:
|
||||
coderay [-language] [input] [-format] [output]
|
||||
|
||||
defaults:
|
||||
language detect from input file name or shebang; fall back to plain text
|
||||
input STDIN
|
||||
format detect from output file name or use terminal; fall back to HTML
|
||||
output STDOUT
|
||||
|
||||
common:
|
||||
coderay file.rb # highlight file to terminal
|
||||
coderay file.rb > file.html # highlight file to HTML page
|
||||
coderay file.rb -div > file.html # highlight file to HTML snippet
|
||||
|
||||
configure output:
|
||||
coderay file.py output.json # output tokens as JSON
|
||||
coderay file.py -loc # count lines of code in Python file
|
||||
|
||||
configure input:
|
||||
coderay -python file # specify the input language
|
||||
coderay -ruby # take input from STDIN
|
||||
|
||||
more:
|
||||
coderay stylesheet [style] # print CSS stylesheet
|
||||
HELP
|
||||
end
|
||||
|
||||
def commands
|
||||
puts <<-COMMANDS
|
||||
general:
|
||||
highlight code highlighting (default command, optional)
|
||||
stylesheet print the CSS stylesheet with the given name (aliases: style, css)
|
||||
|
||||
about:
|
||||
list [of] list all available plugins (or just the scanners|encoders|styles|filetypes)
|
||||
commands print this list
|
||||
help show some help
|
||||
version print CodeRay version
|
||||
COMMANDS
|
||||
end
|
||||
|
||||
def print_list_of plugin_host
|
||||
plugins = plugin_host.all_plugins.map do |plugin|
|
||||
info = " #{plugin.plugin_id}: #{plugin.title}"
|
||||
|
||||
aliases = (plugin.aliases - [:default]).map { |key| "-#{key}" }.sort_by { |key| key.size }
|
||||
if plugin.respond_to?(:file_extension) || !aliases.empty?
|
||||
additional_info = []
|
||||
additional_info << aliases.join(', ') unless aliases.empty?
|
||||
info << " (#{additional_info.join('; ')})"
|
||||
end
|
||||
|
||||
info << ' <-- default' if plugin.aliases.include? :default
|
||||
|
||||
info
|
||||
end
|
||||
puts plugins.sort
|
||||
end
|
||||
|
||||
if option? '-v', '--version'
|
||||
version
|
||||
end
|
||||
|
||||
if option? '-h', '--help'
|
||||
help
|
||||
end
|
||||
|
||||
case subcommand
|
||||
when 'highlight', nil
|
||||
if ARGV.empty?
|
||||
version
|
||||
help
|
||||
else
|
||||
signature = args.map { |arg| arg[/^-/] ? '-' : 'f' }.join
|
||||
names = args.map { |arg| arg.sub(/^-/, '') }
|
||||
case signature
|
||||
when /^$/
|
||||
exit
|
||||
when /^ff?$/
|
||||
input_file, output_file, = *names
|
||||
when /^f-f?$/
|
||||
input_file, output_format, output_file, = *names
|
||||
when /^-ff?$/
|
||||
input_lang, input_file, output_file, = *names
|
||||
when /^-f-f?$/
|
||||
input_lang, input_file, output_format, output_file, = *names
|
||||
when /^--?f?$/
|
||||
input_lang, output_format, output_file, = *names
|
||||
else
|
||||
$stdout = $stderr
|
||||
help
|
||||
puts
|
||||
puts "Unknown parameter order: #{args.join ' '}, expected: [-language] [input] [-format] [output]"
|
||||
exit 1
|
||||
end
|
||||
|
||||
if input_file
|
||||
input_lang ||= CodeRay::FileType.fetch input_file, :text, true
|
||||
end
|
||||
|
||||
if output_file
|
||||
output_format ||= CodeRay::FileType[output_file]
|
||||
else
|
||||
output_format ||= :terminal
|
||||
end
|
||||
|
||||
output_format = :page if output_format.to_s == 'html'
|
||||
|
||||
if input_file
|
||||
input = File.read input_file
|
||||
else
|
||||
input = $stdin.read
|
||||
end
|
||||
|
||||
begin
|
||||
file =
|
||||
if output_file
|
||||
File.open output_file, 'w'
|
||||
else
|
||||
$stdout.sync = true
|
||||
$stdout
|
||||
end
|
||||
CodeRay.encode(input, input_lang, output_format, :out => file)
|
||||
file.puts
|
||||
rescue CodeRay::PluginHost::PluginNotFound => boom
|
||||
$stdout = $stderr
|
||||
if boom.message[/CodeRay::(\w+)s could not load plugin :?(.*?): /]
|
||||
puts "I don't know the #$1 \"#$2\"."
|
||||
else
|
||||
puts boom.message
|
||||
end
|
||||
# puts "I don't know this plugin: #{boom.message[/Could not load plugin (.*?): /, 1]}."
|
||||
rescue CodeRay::Scanners::Scanner::ScanError # FIXME: rescue Errno::EPIPE
|
||||
# this is sometimes raised by pagers; ignore [TODO: wtf?]
|
||||
ensure
|
||||
file.close if output_file
|
||||
end
|
||||
end
|
||||
when 'li', 'list'
|
||||
arg = args.first && args.first.downcase
|
||||
if [nil, 's', 'sc', 'scanner', 'scanners'].include? arg
|
||||
puts 'input languages (Scanners):'
|
||||
print_list_of CodeRay::Scanners
|
||||
end
|
||||
|
||||
if [nil, 'e', 'en', 'enc', 'encoder', 'encoders'].include? arg
|
||||
puts 'output formats (Encoders):'
|
||||
print_list_of CodeRay::Encoders
|
||||
end
|
||||
|
||||
if [nil, 'st', 'style', 'styles'].include? arg
|
||||
puts 'CSS themes for HTML output (Styles):'
|
||||
print_list_of CodeRay::Styles
|
||||
end
|
||||
|
||||
if [nil, 'f', 'ft', 'file', 'filetype', 'filetypes'].include? arg
|
||||
puts 'recognized file types:'
|
||||
|
||||
filetypes = Hash.new { |h, k| h[k] = [] }
|
||||
CodeRay::FileType::TypeFromExt.inject filetypes do |types, (ext, type)|
|
||||
types[type.to_s] << ".#{ext}"
|
||||
types
|
||||
end
|
||||
CodeRay::FileType::TypeFromName.inject filetypes do |types, (name, type)|
|
||||
types[type.to_s] << name
|
||||
types
|
||||
end
|
||||
|
||||
filetypes.sort.each do |type, exts|
|
||||
puts " #{type}: #{exts.sort_by { |ext| ext.size }.join(', ')}"
|
||||
end
|
||||
end
|
||||
when 'stylesheet', 'style', 'css'
|
||||
puts CodeRay::Encoders[:html]::CSS.new(args.first || :default).stylesheet
|
||||
when 'commands'
|
||||
commands
|
||||
when 'help'
|
||||
help
|
||||
else
|
||||
$stdout = $stderr
|
||||
help
|
||||
puts
|
||||
if subcommand[/\A\w+\z/]
|
||||
puts "Unknown command: #{subcommand}"
|
||||
else
|
||||
puts "File not found: #{subcommand}"
|
||||
end
|
||||
exit 1
|
||||
end
|
||||
|
|
@ -0,0 +1,285 @@
|
|||
# encoding: utf-8
|
||||
# Encoding.default_internal = 'UTF-8'
|
||||
|
||||
# = CodeRay Library
|
||||
#
|
||||
# CodeRay is a Ruby library for syntax highlighting.
|
||||
#
|
||||
# I try to make CodeRay easy to use and intuitive, but at the same time fully
|
||||
# featured, complete, fast and efficient.
|
||||
#
|
||||
# See README.
|
||||
#
|
||||
# It consists mainly of
|
||||
# * the main engine: CodeRay (Scanners::Scanner, Tokens, Encoders::Encoder)
|
||||
# * the plugin system: PluginHost, Plugin
|
||||
# * the scanners in CodeRay::Scanners
|
||||
# * the encoders in CodeRay::Encoders
|
||||
# * the styles in CodeRay::Styles
|
||||
#
|
||||
# Here's a fancy graphic to light up this gray docu:
|
||||
#
|
||||
# http://cycnus.de/raindark/coderay/scheme.png
|
||||
#
|
||||
# == Documentation
|
||||
#
|
||||
# See CodeRay, Encoders, Scanners, Tokens.
|
||||
#
|
||||
# == Usage
|
||||
#
|
||||
# Remember you need RubyGems to use CodeRay, unless you have it in your load
|
||||
# path. Run Ruby with -rubygems option if required.
|
||||
#
|
||||
# === Highlight Ruby code in a string as html
|
||||
#
|
||||
# require 'coderay'
|
||||
# print CodeRay.scan('puts "Hello, world!"', :ruby).html
|
||||
#
|
||||
# # prints something like this:
|
||||
# puts <span class="s">"Hello, world!"</span>
|
||||
#
|
||||
#
|
||||
# === Highlight C code from a file in a html div
|
||||
#
|
||||
# require 'coderay'
|
||||
# print CodeRay.scan(File.read('ruby.h'), :c).div
|
||||
# print CodeRay.scan_file('ruby.h').html.div
|
||||
#
|
||||
# You can include this div in your page. The used CSS styles can be printed with
|
||||
#
|
||||
# % coderay_stylesheet
|
||||
#
|
||||
# === Highlight without typing too much
|
||||
#
|
||||
# If you are one of the hasty (or lazy, or extremely curious) people, just run this file:
|
||||
#
|
||||
# % ruby -rubygems /path/to/coderay/coderay.rb > example.html
|
||||
#
|
||||
# and look at the file it created in your browser.
|
||||
#
|
||||
# = CodeRay Module
|
||||
#
|
||||
# The CodeRay module provides convenience methods for the engine.
|
||||
#
|
||||
# * The +lang+ and +format+ arguments select Scanner and Encoder to use. These are
|
||||
# simply lower-case symbols, like <tt>:python</tt> or <tt>:html</tt>.
|
||||
# * All methods take an optional hash as last parameter, +options+, that is send to
|
||||
# the Encoder / Scanner.
|
||||
# * Input and language are always sorted in this order: +code+, +lang+.
|
||||
# (This is in alphabetical order, if you need a mnemonic ;)
|
||||
#
|
||||
# You should be able to highlight everything you want just using these methods;
|
||||
# so there is no need to dive into CodeRay's deep class hierarchy.
|
||||
#
|
||||
# The examples in the demo directory demonstrate common cases using this interface.
|
||||
#
|
||||
# = Basic Access Ways
|
||||
#
|
||||
# Read this to get a general view what CodeRay provides.
|
||||
#
|
||||
# == Scanning
|
||||
#
|
||||
# Scanning means analysing an input string, splitting it up into Tokens.
|
||||
# Each Token knows about what type it is: string, comment, class name, etc.
|
||||
#
|
||||
# Each +lang+ (language) has its own Scanner; for example, <tt>:ruby</tt> code is
|
||||
# handled by CodeRay::Scanners::Ruby.
|
||||
#
|
||||
# CodeRay.scan:: Scan a string in a given language into Tokens.
|
||||
# This is the most common method to use.
|
||||
# CodeRay.scan_file:: Scan a file and guess the language using FileType.
|
||||
#
|
||||
# The Tokens object you get from these methods can encode itself; see Tokens.
|
||||
#
|
||||
# == Encoding
|
||||
#
|
||||
# Encoding means compiling Tokens into an output. This can be colored HTML or
|
||||
# LaTeX, a textual statistic or just the number of non-whitespace tokens.
|
||||
#
|
||||
# Each Encoder provides output in a specific +format+, so you select Encoders via
|
||||
# formats like <tt>:html</tt> or <tt>:statistic</tt>.
|
||||
#
|
||||
# CodeRay.encode:: Scan and encode a string in a given language.
|
||||
# CodeRay.encode_tokens:: Encode the given tokens.
|
||||
# CodeRay.encode_file:: Scan a file, guess the language using FileType and encode it.
|
||||
#
|
||||
# == All-in-One Encoding
|
||||
#
|
||||
# CodeRay.encode:: Highlight a string with a given input and output format.
|
||||
#
|
||||
# == Instanciating
|
||||
#
|
||||
# You can use an Encoder instance to highlight multiple inputs. This way, the setup
|
||||
# for this Encoder must only be done once.
|
||||
#
|
||||
# CodeRay.encoder:: Create an Encoder instance with format and options.
|
||||
# CodeRay.scanner:: Create an Scanner instance for lang, with '' as default code.
|
||||
#
|
||||
# To make use of CodeRay.scanner, use CodeRay::Scanner::code=.
|
||||
#
|
||||
# The scanning methods provide more flexibility; we recommend to use these.
|
||||
#
|
||||
# == Reusing Scanners and Encoders
|
||||
#
|
||||
# If you want to re-use scanners and encoders (because that is faster), see
|
||||
# CodeRay::Duo for the most convenient (and recommended) interface.
|
||||
module CodeRay
|
||||
|
||||
$CODERAY_DEBUG ||= false
|
||||
|
||||
CODERAY_PATH = File.join File.dirname(__FILE__), 'coderay'
|
||||
|
||||
# Assuming the path is a subpath of lib/coderay/
|
||||
def self.coderay_path *path
|
||||
File.join CODERAY_PATH, *path
|
||||
end
|
||||
|
||||
require coderay_path('version')
|
||||
|
||||
# helpers
|
||||
autoload :FileType, coderay_path('helpers', 'file_type')
|
||||
|
||||
# Tokens
|
||||
autoload :Tokens, coderay_path('tokens')
|
||||
autoload :TokensProxy, coderay_path('tokens_proxy')
|
||||
autoload :TokenKinds, coderay_path('token_kinds')
|
||||
|
||||
# Plugin system
|
||||
autoload :PluginHost, coderay_path('helpers', 'plugin')
|
||||
autoload :Plugin, coderay_path('helpers', 'plugin')
|
||||
|
||||
# Plugins
|
||||
autoload :Scanners, coderay_path('scanner')
|
||||
autoload :Encoders, coderay_path('encoder')
|
||||
autoload :Styles, coderay_path('style')
|
||||
|
||||
# convenience access and reusable Encoder/Scanner pair
|
||||
autoload :Duo, coderay_path('duo')
|
||||
|
||||
class << self
|
||||
|
||||
# Scans the given +code+ (a String) with the Scanner for +lang+.
|
||||
#
|
||||
# This is a simple way to use CodeRay. Example:
|
||||
# require 'coderay'
|
||||
# page = CodeRay.scan("puts 'Hello, world!'", :ruby).html
|
||||
#
|
||||
# See also demo/demo_simple.
|
||||
def scan code, lang, options = {}, &block
|
||||
# FIXME: return a proxy for direct-stream encoding
|
||||
TokensProxy.new code, lang, options, block
|
||||
end
|
||||
|
||||
# Scans +filename+ (a path to a code file) with the Scanner for +lang+.
|
||||
#
|
||||
# If +lang+ is :auto or omitted, the CodeRay::FileType module is used to
|
||||
# determine it. If it cannot find out what type it is, it uses
|
||||
# CodeRay::Scanners::Text.
|
||||
#
|
||||
# Calls CodeRay.scan.
|
||||
#
|
||||
# Example:
|
||||
# require 'coderay'
|
||||
# page = CodeRay.scan_file('some_c_code.c').html
|
||||
def scan_file filename, lang = :auto, options = {}, &block
|
||||
lang = FileType.fetch filename, :text, true if lang == :auto
|
||||
code = File.read filename
|
||||
scan code, lang, options, &block
|
||||
end
|
||||
|
||||
# Encode a string.
|
||||
#
|
||||
# This scans +code+ with the the Scanner for +lang+ and then
|
||||
# encodes it with the Encoder for +format+.
|
||||
# +options+ will be passed to the Encoder.
|
||||
#
|
||||
# See CodeRay::Encoder.encode.
|
||||
def encode code, lang, format, options = {}
|
||||
encoder(format, options).encode code, lang, options
|
||||
end
|
||||
|
||||
# Encode pre-scanned Tokens.
|
||||
# Use this together with CodeRay.scan:
|
||||
#
|
||||
# require 'coderay'
|
||||
#
|
||||
# # Highlight a short Ruby code example in a HTML span
|
||||
# tokens = CodeRay.scan '1 + 2', :ruby
|
||||
# puts CodeRay.encode_tokens(tokens, :span)
|
||||
#
|
||||
def encode_tokens tokens, format, options = {}
|
||||
encoder(format, options).encode_tokens tokens, options
|
||||
end
|
||||
|
||||
# Encodes +filename+ (a path to a code file) with the Scanner for +lang+.
|
||||
#
|
||||
# See CodeRay.scan_file.
|
||||
# Notice that the second argument is the output +format+, not the input language.
|
||||
#
|
||||
# Example:
|
||||
# require 'coderay'
|
||||
# page = CodeRay.encode_file 'some_c_code.c', :html
|
||||
def encode_file filename, format, options = {}
|
||||
tokens = scan_file filename, :auto, get_scanner_options(options)
|
||||
encode_tokens tokens, format, options
|
||||
end
|
||||
|
||||
# Highlight a string into a HTML <div>.
|
||||
#
|
||||
# CSS styles use classes, so you have to include a stylesheet
|
||||
# in your output.
|
||||
#
|
||||
# See encode.
|
||||
def highlight code, lang, options = { :css => :class }, format = :div
|
||||
encode code, lang, format, options
|
||||
end
|
||||
|
||||
# Highlight a file into a HTML <div>.
|
||||
#
|
||||
# CSS styles use classes, so you have to include a stylesheet
|
||||
# in your output.
|
||||
#
|
||||
# See encode.
|
||||
def highlight_file filename, options = { :css => :class }, format = :div
|
||||
encode_file filename, format, options
|
||||
end
|
||||
|
||||
# Finds the Encoder class for +format+ and creates an instance, passing
|
||||
# +options+ to it.
|
||||
#
|
||||
# Example:
|
||||
# require 'coderay'
|
||||
#
|
||||
# stats = CodeRay.encoder(:statistic)
|
||||
# stats.encode("puts 17 + 4\n", :ruby)
|
||||
#
|
||||
# puts '%d out of %d tokens have the kind :integer.' % [
|
||||
# stats.type_stats[:integer].count,
|
||||
# stats.real_token_count
|
||||
# ]
|
||||
# #-> 2 out of 4 tokens have the kind :integer.
|
||||
def encoder format, options = {}
|
||||
Encoders[format].new options
|
||||
end
|
||||
|
||||
# Finds the Scanner class for +lang+ and creates an instance, passing
|
||||
# +options+ to it.
|
||||
#
|
||||
# See Scanner.new.
|
||||
def scanner lang, options = {}, &block
|
||||
Scanners[lang].new '', options, &block
|
||||
end
|
||||
|
||||
# Extract the options for the scanner from the +options+ hash.
|
||||
#
|
||||
# Returns an empty Hash if <tt>:scanner_options</tt> is not set.
|
||||
#
|
||||
# This is used if a method like CodeRay.encode has to provide options
|
||||
# for Encoder _and_ scanner.
|
||||
def get_scanner_options options
|
||||
options.fetch :scanner_options, {}
|
||||
end
|
||||
|
||||
end
|
||||
|
||||
end
|
||||
|
|
@ -0,0 +1,81 @@
|
|||
module CodeRay
|
||||
|
||||
# = Duo
|
||||
#
|
||||
# A Duo is a convenient way to use CodeRay. You just create a Duo,
|
||||
# giving it a lang (language of the input code) and a format (desired
|
||||
# output format), and call Duo#highlight with the code.
|
||||
#
|
||||
# Duo makes it easy to re-use both scanner and encoder for a repetitive
|
||||
# task. It also provides a very easy interface syntax:
|
||||
#
|
||||
# require 'coderay'
|
||||
# CodeRay::Duo[:python, :div].highlight 'import this'
|
||||
#
|
||||
# Until you want to do uncommon things with CodeRay, I recommend to use
|
||||
# this method, since it takes care of everything.
|
||||
class Duo
|
||||
|
||||
attr_accessor :lang, :format, :options
|
||||
|
||||
# Create a new Duo, holding a lang and a format to highlight code.
|
||||
#
|
||||
# simple:
|
||||
# CodeRay::Duo[:ruby, :html].highlight 'bla 42'
|
||||
#
|
||||
# with options:
|
||||
# CodeRay::Duo[:ruby, :html, :hint => :debug].highlight '????::??'
|
||||
#
|
||||
# alternative syntax without options:
|
||||
# CodeRay::Duo[:ruby => :statistic].encode 'class << self; end'
|
||||
#
|
||||
# alternative syntax with options:
|
||||
# CodeRay::Duo[{ :ruby => :statistic }, :do => :something].encode 'abc'
|
||||
#
|
||||
# The options are forwarded to scanner and encoder
|
||||
# (see CodeRay.get_scanner_options).
|
||||
def initialize lang = nil, format = nil, options = {}
|
||||
if format.nil? && lang.is_a?(Hash) && lang.size == 1
|
||||
@lang = lang.keys.first
|
||||
@format = lang[@lang]
|
||||
else
|
||||
@lang = lang
|
||||
@format = format
|
||||
end
|
||||
@options = options
|
||||
end
|
||||
|
||||
class << self
|
||||
# To allow calls like Duo[:ruby, :html].highlight.
|
||||
alias [] new
|
||||
end
|
||||
|
||||
# The scanner of the duo. Only created once.
|
||||
def scanner
|
||||
@scanner ||= CodeRay.scanner @lang, CodeRay.get_scanner_options(@options)
|
||||
end
|
||||
|
||||
# The encoder of the duo. Only created once.
|
||||
def encoder
|
||||
@encoder ||= CodeRay.encoder @format, @options
|
||||
end
|
||||
|
||||
# Tokenize and highlight the code using +scanner+ and +encoder+.
|
||||
def encode code, options = {}
|
||||
options = @options.merge options
|
||||
encoder.encode(code, @lang, options)
|
||||
end
|
||||
alias highlight encode
|
||||
|
||||
# Allows to use Duo like a proc object:
|
||||
#
|
||||
# CodeRay::Duo[:python => :yaml].call(code)
|
||||
#
|
||||
# or, in Ruby 1.9 and later:
|
||||
#
|
||||
# CodeRay::Duo[:python => :yaml].(code)
|
||||
alias call encode
|
||||
|
||||
end
|
||||
|
||||
end
|
||||
|
|
@ -0,0 +1,201 @@
|
|||
module CodeRay
|
||||
|
||||
# This module holds the Encoder class and its subclasses.
|
||||
# For example, the HTML encoder is named CodeRay::Encoders::HTML
|
||||
# can be found in coderay/encoders/html.
|
||||
#
|
||||
# Encoders also provides methods and constants for the register
|
||||
# mechanism and the [] method that returns the Encoder class
|
||||
# belonging to the given format.
|
||||
module Encoders
|
||||
|
||||
extend PluginHost
|
||||
plugin_path File.dirname(__FILE__), 'encoders'
|
||||
|
||||
# = Encoder
|
||||
#
|
||||
# The Encoder base class. Together with Scanner and
|
||||
# Tokens, it forms the highlighting triad.
|
||||
#
|
||||
# Encoder instances take a Tokens object and do something with it.
|
||||
#
|
||||
# The most common Encoder is surely the HTML encoder
|
||||
# (CodeRay::Encoders::HTML). It highlights the code in a colorful
|
||||
# html page.
|
||||
# If you want the highlighted code in a div or a span instead,
|
||||
# use its subclasses Div and Span.
|
||||
class Encoder
|
||||
extend Plugin
|
||||
plugin_host Encoders
|
||||
|
||||
class << self
|
||||
|
||||
# If FILE_EXTENSION isn't defined, this method returns the
|
||||
# downcase class name instead.
|
||||
def const_missing sym
|
||||
if sym == :FILE_EXTENSION
|
||||
(defined?(@plugin_id) && @plugin_id || name[/\w+$/].downcase).to_s
|
||||
else
|
||||
super
|
||||
end
|
||||
end
|
||||
|
||||
# The default file extension for output file of this encoder class.
|
||||
def file_extension
|
||||
self::FILE_EXTENSION
|
||||
end
|
||||
|
||||
end
|
||||
|
||||
# Subclasses are to store their default options in this constant.
|
||||
DEFAULT_OPTIONS = { }
|
||||
|
||||
# The options you gave the Encoder at creating.
|
||||
attr_accessor :options, :scanner
|
||||
|
||||
# Creates a new Encoder.
|
||||
# +options+ is saved and used for all encode operations, as long
|
||||
# as you don't overwrite it there by passing additional options.
|
||||
#
|
||||
# Encoder objects provide three encode methods:
|
||||
# - encode simply takes a +code+ string and a +lang+
|
||||
# - encode_tokens expects a +tokens+ object instead
|
||||
#
|
||||
# Each method has an optional +options+ parameter. These are
|
||||
# added to the options you passed at creation.
|
||||
def initialize options = {}
|
||||
@options = self.class::DEFAULT_OPTIONS.merge options
|
||||
@@CODERAY_TOKEN_INTERFACE_DEPRECATION_WARNING_GIVEN = false
|
||||
end
|
||||
|
||||
# Encode a Tokens object.
|
||||
def encode_tokens tokens, options = {}
|
||||
options = @options.merge options
|
||||
@scanner = tokens.scanner if tokens.respond_to? :scanner
|
||||
setup options
|
||||
compile tokens, options
|
||||
finish options
|
||||
end
|
||||
|
||||
# Encode the given +code+ using the Scanner for +lang+.
|
||||
def encode code, lang, options = {}
|
||||
options = @options.merge options
|
||||
@scanner = Scanners[lang].new code, CodeRay.get_scanner_options(options).update(:tokens => self)
|
||||
setup options
|
||||
@scanner.tokenize
|
||||
finish options
|
||||
end
|
||||
|
||||
# You can use highlight instead of encode, if that seems
|
||||
# more clear to you.
|
||||
alias highlight encode
|
||||
|
||||
# The default file extension for this encoder.
|
||||
def file_extension
|
||||
self.class.file_extension
|
||||
end
|
||||
|
||||
def << token
|
||||
unless @@CODERAY_TOKEN_INTERFACE_DEPRECATION_WARNING_GIVEN
|
||||
warn 'Using old Tokens#<< interface.'
|
||||
@@CODERAY_TOKEN_INTERFACE_DEPRECATION_WARNING_GIVEN = true
|
||||
end
|
||||
self.token(*token)
|
||||
end
|
||||
|
||||
# Called with +content+ and +kind+ of the currently scanned token.
|
||||
# For simple scanners, it's enougth to implement this method.
|
||||
#
|
||||
# By default, it calls text_token, begin_group, end_group, begin_line,
|
||||
# or end_line, depending on the +content+.
|
||||
def token content, kind
|
||||
case content
|
||||
when String
|
||||
text_token content, kind
|
||||
when :begin_group
|
||||
begin_group kind
|
||||
when :end_group
|
||||
end_group kind
|
||||
when :begin_line
|
||||
begin_line kind
|
||||
when :end_line
|
||||
end_line kind
|
||||
else
|
||||
raise ArgumentError, 'Unknown token content type: %p, kind = %p' % [content, kind]
|
||||
end
|
||||
end
|
||||
|
||||
# Called for each text token ([text, kind]), where text is a String.
|
||||
def text_token text, kind
|
||||
@out << text
|
||||
end
|
||||
|
||||
# Starts a token group with the given +kind+.
|
||||
def begin_group kind
|
||||
end
|
||||
|
||||
# Ends a token group with the given +kind+.
|
||||
def end_group kind
|
||||
end
|
||||
|
||||
# Starts a new line token group with the given +kind+.
|
||||
def begin_line kind
|
||||
end
|
||||
|
||||
# Ends a new line token group with the given +kind+.
|
||||
def end_line kind
|
||||
end
|
||||
|
||||
protected
|
||||
|
||||
# Called with merged options before encoding starts.
|
||||
# Sets @out to an empty string.
|
||||
#
|
||||
# See the HTML Encoder for an example of option caching.
|
||||
def setup options
|
||||
@out = get_output(options)
|
||||
end
|
||||
|
||||
def get_output options
|
||||
options[:out] || ''
|
||||
end
|
||||
|
||||
# Append data.to_s to the output. Returns the argument.
|
||||
def output data
|
||||
@out << data.to_s
|
||||
data
|
||||
end
|
||||
|
||||
# Called with merged options after encoding starts.
|
||||
# The return value is the result of encoding, typically @out.
|
||||
def finish options
|
||||
@out
|
||||
end
|
||||
|
||||
# Do the encoding.
|
||||
#
|
||||
# The already created +tokens+ object must be used; it must be a
|
||||
# Tokens object.
|
||||
def compile tokens, options = {}
|
||||
content = nil
|
||||
for item in tokens
|
||||
if item.is_a? Array
|
||||
raise ArgumentError, 'Two-element array tokens are no longer supported.'
|
||||
end
|
||||
if content
|
||||
token content, item
|
||||
content = nil
|
||||
else
|
||||
content = item
|
||||
end
|
||||
end
|
||||
raise 'odd number list for Tokens' if content
|
||||
end
|
||||
|
||||
alias tokens compile
|
||||
public :tokens
|
||||
|
||||
end
|
||||
|
||||
end
|
||||
end
|
||||
|
|
@ -0,0 +1,17 @@
|
|||
module CodeRay
|
||||
module Encoders
|
||||
|
||||
map \
|
||||
:loc => :lines_of_code,
|
||||
:plain => :text,
|
||||
:plaintext => :text,
|
||||
:remove_comments => :comment_filter,
|
||||
:stats => :statistic,
|
||||
:term => :terminal,
|
||||
:tty => :terminal,
|
||||
:yml => :yaml
|
||||
|
||||
# No default because Tokens#nonsense should raise NoMethodError.
|
||||
|
||||
end
|
||||
end
|
||||
|
|
@ -0,0 +1,25 @@
|
|||
module CodeRay
|
||||
module Encoders
|
||||
|
||||
load :token_kind_filter
|
||||
|
||||
# A simple Filter that removes all tokens of the :comment kind.
|
||||
#
|
||||
# Alias: +remove_comments+
|
||||
#
|
||||
# Usage:
|
||||
# CodeRay.scan('print # foo', :ruby).comment_filter.text
|
||||
# #-> "print "
|
||||
#
|
||||
# See also: TokenKindFilter, LinesOfCode
|
||||
class CommentFilter < TokenKindFilter
|
||||
|
||||
register_for :comment_filter
|
||||
|
||||
DEFAULT_OPTIONS = superclass::DEFAULT_OPTIONS.merge \
|
||||
:exclude => [:comment, :docstring]
|
||||
|
||||
end
|
||||
|
||||
end
|
||||
end
|
||||
|
|
@ -0,0 +1,39 @@
|
|||
module CodeRay
|
||||
module Encoders
|
||||
|
||||
# Returns the number of tokens.
|
||||
#
|
||||
# Text and block tokens are counted.
|
||||
class Count < Encoder
|
||||
|
||||
register_for :count
|
||||
|
||||
protected
|
||||
|
||||
def setup options
|
||||
super
|
||||
|
||||
@count = 0
|
||||
end
|
||||
|
||||
def finish options
|
||||
output @count
|
||||
end
|
||||
|
||||
public
|
||||
|
||||
def text_token text, kind
|
||||
@count += 1
|
||||
end
|
||||
|
||||
def begin_group kind
|
||||
@count += 1
|
||||
end
|
||||
alias end_group begin_group
|
||||
alias begin_line begin_group
|
||||
alias end_line begin_group
|
||||
|
||||
end
|
||||
|
||||
end
|
||||
end
|
||||
|
|
@ -0,0 +1,61 @@
|
|||
module CodeRay
|
||||
module Encoders
|
||||
|
||||
# = Debug Encoder
|
||||
#
|
||||
# Fast encoder producing simple debug output.
|
||||
#
|
||||
# It is readable and diff-able and is used for testing.
|
||||
#
|
||||
# You cannot fully restore the tokens information from the
|
||||
# output, because consecutive :space tokens are merged.
|
||||
# Use Tokens#dump for caching purposes.
|
||||
#
|
||||
# See also: Scanners::Debug
|
||||
class Debug < Encoder
|
||||
|
||||
register_for :debug
|
||||
|
||||
FILE_EXTENSION = 'raydebug'
|
||||
|
||||
def initialize options = {}
|
||||
super
|
||||
@opened = []
|
||||
end
|
||||
|
||||
def text_token text, kind
|
||||
if kind == :space
|
||||
@out << text
|
||||
else
|
||||
# TODO: Escape (
|
||||
text = text.gsub(/[)\\]/, '\\\\\0') # escape ) and \
|
||||
@out << kind.to_s << '(' << text << ')'
|
||||
end
|
||||
end
|
||||
|
||||
def begin_group kind
|
||||
@opened << kind
|
||||
@out << kind.to_s << '<'
|
||||
end
|
||||
|
||||
def end_group kind
|
||||
if @opened.last != kind
|
||||
puts @out
|
||||
raise "we are inside #{@opened.inspect}, not #{kind}"
|
||||
end
|
||||
@opened.pop
|
||||
@out << '>'
|
||||
end
|
||||
|
||||
def begin_line kind
|
||||
@out << kind.to_s << '['
|
||||
end
|
||||
|
||||
def end_line kind
|
||||
@out << ']'
|
||||
end
|
||||
|
||||
end
|
||||
|
||||
end
|
||||
end
|
||||
|
|
@ -0,0 +1,23 @@
|
|||
module CodeRay
|
||||
module Encoders
|
||||
|
||||
load :html
|
||||
|
||||
# Wraps HTML output into a DIV element, using inline styles by default.
|
||||
#
|
||||
# See Encoders::HTML for available options.
|
||||
class Div < HTML
|
||||
|
||||
FILE_EXTENSION = 'div.html'
|
||||
|
||||
register_for :div
|
||||
|
||||
DEFAULT_OPTIONS = HTML::DEFAULT_OPTIONS.merge \
|
||||
:css => :style,
|
||||
:wrap => :div,
|
||||
:line_numbers => false
|
||||
|
||||
end
|
||||
|
||||
end
|
||||
end
|
||||
|
|
@ -0,0 +1,58 @@
|
|||
module CodeRay
|
||||
module Encoders
|
||||
|
||||
# A Filter encoder has another Tokens instance as output.
|
||||
# It can be subclass to select, remove, or modify tokens in the stream.
|
||||
#
|
||||
# Subclasses of Filter are called "Filters" and can be chained.
|
||||
#
|
||||
# == Options
|
||||
#
|
||||
# === :tokens
|
||||
#
|
||||
# The Tokens object which will receive the output.
|
||||
#
|
||||
# Default: Tokens.new
|
||||
#
|
||||
# See also: TokenKindFilter
|
||||
class Filter < Encoder
|
||||
|
||||
register_for :filter
|
||||
|
||||
protected
|
||||
def setup options
|
||||
super
|
||||
|
||||
@tokens = options[:tokens] || Tokens.new
|
||||
end
|
||||
|
||||
def finish options
|
||||
output @tokens
|
||||
end
|
||||
|
||||
public
|
||||
|
||||
def text_token text, kind # :nodoc:
|
||||
@tokens.text_token text, kind
|
||||
end
|
||||
|
||||
def begin_group kind # :nodoc:
|
||||
@tokens.begin_group kind
|
||||
end
|
||||
|
||||
def begin_line kind # :nodoc:
|
||||
@tokens.begin_line kind
|
||||
end
|
||||
|
||||
def end_group kind # :nodoc:
|
||||
@tokens.end_group kind
|
||||
end
|
||||
|
||||
def end_line kind # :nodoc:
|
||||
@tokens.end_line kind
|
||||
end
|
||||
|
||||
end
|
||||
|
||||
end
|
||||
end
|
||||
|
|
@ -0,0 +1,327 @@
|
|||
require 'set'
|
||||
|
||||
module CodeRay
|
||||
module Encoders
|
||||
|
||||
# = HTML Encoder
|
||||
#
|
||||
# This is CodeRay's most important highlighter:
|
||||
# It provides save, fast XHTML generation and CSS support.
|
||||
#
|
||||
# == Usage
|
||||
#
|
||||
# require 'coderay'
|
||||
# puts CodeRay.scan('Some /code/', :ruby).html #-> a HTML page
|
||||
# puts CodeRay.scan('Some /code/', :ruby).html(:wrap => :span)
|
||||
# #-> <span class="CodeRay"><span class="co">Some</span> /code/</span>
|
||||
# puts CodeRay.scan('Some /code/', :ruby).span #-> the same
|
||||
#
|
||||
# puts CodeRay.scan('Some code', :ruby).html(
|
||||
# :wrap => nil,
|
||||
# :line_numbers => :inline,
|
||||
# :css => :style
|
||||
# )
|
||||
#
|
||||
# == Options
|
||||
#
|
||||
# === :tab_width
|
||||
# Convert \t characters to +n+ spaces (a number.)
|
||||
#
|
||||
# Default: 8
|
||||
#
|
||||
# === :css
|
||||
# How to include the styles; can be :class or :style.
|
||||
#
|
||||
# Default: :class
|
||||
#
|
||||
# === :wrap
|
||||
# Wrap in :page, :div, :span or nil.
|
||||
#
|
||||
# You can also use Encoders::Div and Encoders::Span.
|
||||
#
|
||||
# Default: nil
|
||||
#
|
||||
# === :title
|
||||
#
|
||||
# The title of the HTML page (works only when :wrap is set to :page.)
|
||||
#
|
||||
# Default: 'CodeRay output'
|
||||
#
|
||||
# === :break_lines
|
||||
#
|
||||
# Split multiline blocks at line breaks.
|
||||
# Forced to true if :line_numbers option is set to :inline.
|
||||
#
|
||||
# Default: false
|
||||
#
|
||||
# === :line_numbers
|
||||
# Include line numbers in :table, :inline, or nil (no line numbers)
|
||||
#
|
||||
# Default: nil
|
||||
#
|
||||
# === :line_number_anchors
|
||||
# Adds anchors and links to the line numbers. Can be false (off), true (on),
|
||||
# or a prefix string that will be prepended to the anchor name.
|
||||
#
|
||||
# The prefix must consist only of letters, digits, and underscores.
|
||||
#
|
||||
# Default: true, default prefix name: "line"
|
||||
#
|
||||
# === :line_number_start
|
||||
# Where to start with line number counting.
|
||||
#
|
||||
# Default: 1
|
||||
#
|
||||
# === :bold_every
|
||||
# Make every +n+-th number appear bold.
|
||||
#
|
||||
# Default: 10
|
||||
#
|
||||
# === :highlight_lines
|
||||
#
|
||||
# Highlights certain line numbers.
|
||||
# Can be any Enumerable, typically just an Array or Range, of numbers.
|
||||
#
|
||||
# Bolding is deactivated when :highlight_lines is set. It only makes sense
|
||||
# in combination with :line_numbers.
|
||||
#
|
||||
# Default: nil
|
||||
#
|
||||
# === :hint
|
||||
# Include some information into the output using the title attribute.
|
||||
# Can be :info (show token kind on mouse-over), :info_long (with full path)
|
||||
# or :debug (via inspect).
|
||||
#
|
||||
# Default: false
|
||||
class HTML < Encoder
|
||||
|
||||
register_for :html
|
||||
|
||||
FILE_EXTENSION = 'snippet.html'
|
||||
|
||||
DEFAULT_OPTIONS = {
|
||||
:tab_width => 8,
|
||||
|
||||
:css => :class,
|
||||
:style => :alpha,
|
||||
:wrap => nil,
|
||||
:title => 'CodeRay output',
|
||||
|
||||
:break_lines => false,
|
||||
|
||||
:line_numbers => nil,
|
||||
:line_number_anchors => 'n',
|
||||
:line_number_start => 1,
|
||||
:bold_every => 10,
|
||||
:highlight_lines => nil,
|
||||
|
||||
:hint => false,
|
||||
}
|
||||
|
||||
autoload :Output, CodeRay.coderay_path('encoders', 'html', 'output')
|
||||
autoload :CSS, CodeRay.coderay_path('encoders', 'html', 'css')
|
||||
autoload :Numbering, CodeRay.coderay_path('encoders', 'html', 'numbering')
|
||||
|
||||
attr_reader :css
|
||||
|
||||
protected
|
||||
|
||||
HTML_ESCAPE = { #:nodoc:
|
||||
'&' => '&',
|
||||
'"' => '"',
|
||||
'>' => '>',
|
||||
'<' => '<',
|
||||
}
|
||||
|
||||
# This was to prevent illegal HTML.
|
||||
# Strange chars should still be avoided in codes.
|
||||
evil_chars = Array(0x00...0x20) - [?\n, ?\t, ?\s]
|
||||
evil_chars.each { |i| HTML_ESCAPE[i.chr] = ' ' }
|
||||
#ansi_chars = Array(0x7f..0xff)
|
||||
#ansi_chars.each { |i| HTML_ESCAPE[i.chr] = '&#%d;' % i }
|
||||
# \x9 (\t) and \xA (\n) not included
|
||||
#HTML_ESCAPE_PATTERN = /[\t&"><\0-\x8\xB-\x1f\x7f-\xff]/
|
||||
HTML_ESCAPE_PATTERN = /[\t"&><\0-\x8\xB-\x1f]/
|
||||
|
||||
TOKEN_KIND_TO_INFO = Hash.new do |h, kind|
|
||||
h[kind] = kind.to_s.gsub(/_/, ' ').gsub(/\b\w/) { $&.capitalize }
|
||||
end
|
||||
|
||||
TRANSPARENT_TOKEN_KINDS = Set[
|
||||
:delimiter, :modifier, :content, :escape, :inline_delimiter,
|
||||
]
|
||||
|
||||
# Generate a hint about the given +kinds+ in a +hint+ style.
|
||||
#
|
||||
# +hint+ may be :info, :info_long or :debug.
|
||||
def self.token_path_to_hint hint, kinds
|
||||
kinds = Array kinds
|
||||
title =
|
||||
case hint
|
||||
when :info
|
||||
kinds = kinds[1..-1] if TRANSPARENT_TOKEN_KINDS.include? kinds.first
|
||||
TOKEN_KIND_TO_INFO[kinds.first]
|
||||
when :info_long
|
||||
kinds.reverse.map { |kind| TOKEN_KIND_TO_INFO[kind] }.join('/')
|
||||
when :debug
|
||||
kinds.inspect
|
||||
end
|
||||
title ? " title=\"#{title}\"" : ''
|
||||
end
|
||||
|
||||
def setup options
|
||||
super
|
||||
|
||||
if options[:wrap] || options[:line_numbers]
|
||||
@real_out = @out
|
||||
@out = ''
|
||||
end
|
||||
|
||||
options[:break_lines] = true if options[:line_numbers] == :inline
|
||||
|
||||
@break_lines = (options[:break_lines] == true)
|
||||
|
||||
@HTML_ESCAPE = HTML_ESCAPE.dup
|
||||
@HTML_ESCAPE["\t"] = ' ' * options[:tab_width]
|
||||
|
||||
@opened = []
|
||||
@last_opened = nil
|
||||
@css = CSS.new options[:style]
|
||||
|
||||
hint = options[:hint]
|
||||
if hint && ![:debug, :info, :info_long].include?(hint)
|
||||
raise ArgumentError, "Unknown value %p for :hint; \
|
||||
expected :info, :info_long, :debug, false, or nil." % hint
|
||||
end
|
||||
|
||||
css_classes = TokenKinds
|
||||
case options[:css]
|
||||
when :class
|
||||
@span_for_kind = Hash.new do |h, k|
|
||||
if k.is_a? ::Symbol
|
||||
kind = k_dup = k
|
||||
else
|
||||
kind = k.first
|
||||
k_dup = k.dup
|
||||
end
|
||||
if kind != :space && (hint || css_class = css_classes[kind])
|
||||
title = HTML.token_path_to_hint hint, k if hint
|
||||
css_class ||= css_classes[kind]
|
||||
h[k_dup] = "<span#{title}#{" class=\"#{css_class}\"" if css_class}>"
|
||||
else
|
||||
h[k_dup] = nil
|
||||
end
|
||||
end
|
||||
when :style
|
||||
@span_for_kind = Hash.new do |h, k|
|
||||
kind = k.is_a?(Symbol) ? k : k.first
|
||||
h[k.is_a?(Symbol) ? k : k.dup] =
|
||||
if kind != :space && (hint || css_classes[kind])
|
||||
title = HTML.token_path_to_hint hint, k if hint
|
||||
style = @css.get_style Array(k).map { |c| css_classes[c] }
|
||||
"<span#{title}#{" style=\"#{style}\"" if style}>"
|
||||
end
|
||||
end
|
||||
else
|
||||
raise ArgumentError, "Unknown value %p for :css." % options[:css]
|
||||
end
|
||||
|
||||
@set_last_opened = options[:hint] || options[:css] == :style
|
||||
end
|
||||
|
||||
def finish options
|
||||
unless @opened.empty?
|
||||
warn '%d tokens still open: %p' % [@opened.size, @opened] if $CODERAY_DEBUG
|
||||
@out << '</span>' while @opened.pop
|
||||
@last_opened = nil
|
||||
end
|
||||
|
||||
@out.extend Output
|
||||
@out.css = @css
|
||||
if options[:line_numbers]
|
||||
Numbering.number! @out, options[:line_numbers], options
|
||||
end
|
||||
@out.wrap! options[:wrap]
|
||||
@out.apply_title! options[:title]
|
||||
|
||||
if defined?(@real_out) && @real_out
|
||||
@real_out << @out
|
||||
@out = @real_out
|
||||
end
|
||||
|
||||
super
|
||||
end
|
||||
|
||||
public
|
||||
|
||||
def text_token text, kind
|
||||
if text =~ /#{HTML_ESCAPE_PATTERN}/o
|
||||
text = text.gsub(/#{HTML_ESCAPE_PATTERN}/o) { |m| @HTML_ESCAPE[m] }
|
||||
end
|
||||
|
||||
style = @span_for_kind[@last_opened ? [kind, *@opened] : kind]
|
||||
|
||||
if @break_lines && (i = text.index("\n")) && (c = @opened.size + (style ? 1 : 0)) > 0
|
||||
close = '</span>' * c
|
||||
reopen = ''
|
||||
@opened.each_with_index do |k, index|
|
||||
reopen << (@span_for_kind[index > 0 ? [k, *@opened[0 ... index ]] : k] || '<span>')
|
||||
end
|
||||
text[i .. -1] = text[i .. -1].gsub("\n", "#{close}\n#{reopen}#{style}")
|
||||
end
|
||||
|
||||
if style
|
||||
@out << style << text << '</span>'
|
||||
else
|
||||
@out << text
|
||||
end
|
||||
end
|
||||
|
||||
# token groups, eg. strings
|
||||
def begin_group kind
|
||||
@out << (@span_for_kind[@last_opened ? [kind, *@opened] : kind] || '<span>')
|
||||
@opened << kind
|
||||
@last_opened = kind if @set_last_opened
|
||||
end
|
||||
|
||||
def end_group kind
|
||||
if $CODERAY_DEBUG && (@opened.empty? || @opened.last != kind)
|
||||
warn 'Malformed token stream: Trying to close a token (%p) ' \
|
||||
'that is not open. Open are: %p.' % [kind, @opened[1..-1]]
|
||||
end
|
||||
if @opened.pop
|
||||
@out << '</span>'
|
||||
@last_opened = @opened.last if @last_opened
|
||||
end
|
||||
end
|
||||
|
||||
# whole lines to be highlighted, eg. a deleted line in a diff
|
||||
def begin_line kind
|
||||
if style = @span_for_kind[@last_opened ? [kind, *@opened] : kind]
|
||||
if style['class="']
|
||||
@out << style.sub('class="', 'class="line ')
|
||||
else
|
||||
@out << style.sub('>', ' class="line">')
|
||||
end
|
||||
else
|
||||
@out << '<span class="line">'
|
||||
end
|
||||
@opened << kind
|
||||
@last_opened = kind if @options[:css] == :style
|
||||
end
|
||||
|
||||
def end_line kind
|
||||
if $CODERAY_DEBUG && (@opened.empty? || @opened.last != kind)
|
||||
warn 'Malformed token stream: Trying to close a line (%p) ' \
|
||||
'that is not open. Open are: %p.' % [kind, @opened[1..-1]]
|
||||
end
|
||||
if @opened.pop
|
||||
@out << '</span>'
|
||||
@last_opened = @opened.last if @last_opened
|
||||
end
|
||||
end
|
||||
|
||||
end
|
||||
|
||||
end
|
||||
end
|
||||
|
|
@ -0,0 +1,65 @@
|
|||
module CodeRay
|
||||
module Encoders
|
||||
|
||||
class HTML
|
||||
class CSS # :nodoc:
|
||||
|
||||
attr :stylesheet
|
||||
|
||||
def CSS.load_stylesheet style = nil
|
||||
CodeRay::Styles[style]
|
||||
end
|
||||
|
||||
def initialize style = :default
|
||||
@classes = Hash.new
|
||||
style = CSS.load_stylesheet style
|
||||
@stylesheet = [
|
||||
style::CSS_MAIN_STYLES,
|
||||
style::TOKEN_COLORS.gsub(/^(?!$)/, '.CodeRay ')
|
||||
].join("\n")
|
||||
parse style::TOKEN_COLORS
|
||||
end
|
||||
|
||||
def get_style styles
|
||||
cl = @classes[styles.first]
|
||||
return '' unless cl
|
||||
style = ''
|
||||
1.upto styles.size do |offset|
|
||||
break if style = cl[styles[offset .. -1]]
|
||||
end
|
||||
# warn 'Style not found: %p' % [styles] if style.empty?
|
||||
return style
|
||||
end
|
||||
|
||||
private
|
||||
|
||||
CSS_CLASS_PATTERN = /
|
||||
( # $1 = selectors
|
||||
(?:
|
||||
(?: \s* \. [-\w]+ )+
|
||||
\s* ,?
|
||||
)+
|
||||
)
|
||||
\s* \{ \s*
|
||||
( [^\}]+ )? # $2 = style
|
||||
\s* \} \s*
|
||||
|
|
||||
( [^\n]+ ) # $3 = error
|
||||
/mx
|
||||
def parse stylesheet
|
||||
stylesheet.scan CSS_CLASS_PATTERN do |selectors, style, error|
|
||||
raise "CSS parse error: '#{error.inspect}' not recognized" if error
|
||||
for selector in selectors.split(',')
|
||||
classes = selector.scan(/[-\w]+/)
|
||||
cl = classes.pop
|
||||
@classes[cl] ||= Hash.new
|
||||
@classes[cl][classes] = style.to_s.strip.delete(' ').chomp(';')
|
||||
end
|
||||
end
|
||||
end
|
||||
|
||||
end
|
||||
end
|
||||
|
||||
end
|
||||
end
|
||||
|
|
@ -0,0 +1,103 @@
|
|||
module CodeRay
|
||||
module Encoders
|
||||
|
||||
class HTML
|
||||
|
||||
module Numbering # :nodoc:
|
||||
|
||||
def self.number! output, mode = :table, options = {}
|
||||
return self unless mode
|
||||
|
||||
options = DEFAULT_OPTIONS.merge options
|
||||
|
||||
start = options[:line_number_start]
|
||||
unless start.is_a? Integer
|
||||
raise ArgumentError, "Invalid value %p for :line_number_start; Integer expected." % start
|
||||
end
|
||||
|
||||
anchor_prefix = options[:line_number_anchors]
|
||||
anchor_prefix = 'line' if anchor_prefix == true
|
||||
anchor_prefix = anchor_prefix.to_s[/\w+/] if anchor_prefix
|
||||
anchoring =
|
||||
if anchor_prefix
|
||||
proc do |line|
|
||||
line = line.to_s
|
||||
anchor = anchor_prefix + line
|
||||
"<a href=\"##{anchor}\" name=\"#{anchor}\">#{line}</a>"
|
||||
end
|
||||
else
|
||||
proc { |line| line.to_s } # :to_s.to_proc in Ruby 1.8.7+
|
||||
end
|
||||
|
||||
bold_every = options[:bold_every]
|
||||
highlight_lines = options[:highlight_lines]
|
||||
bolding =
|
||||
if bold_every == false && highlight_lines == nil
|
||||
anchoring
|
||||
elsif highlight_lines.is_a? Enumerable
|
||||
highlight_lines = highlight_lines.to_set
|
||||
proc do |line|
|
||||
if highlight_lines.include? line
|
||||
"<strong class=\"highlighted\">#{anchoring[line]}</strong>" # highlighted line numbers in bold
|
||||
else
|
||||
anchoring[line]
|
||||
end
|
||||
end
|
||||
elsif bold_every.is_a? Integer
|
||||
raise ArgumentError, ":bolding can't be 0." if bold_every == 0
|
||||
proc do |line|
|
||||
if line % bold_every == 0
|
||||
"<strong>#{anchoring[line]}</strong>" # every bold_every-th number in bold
|
||||
else
|
||||
anchoring[line]
|
||||
end
|
||||
end
|
||||
else
|
||||
raise ArgumentError, 'Invalid value %p for :bolding; false or Integer expected.' % bold_every
|
||||
end
|
||||
|
||||
line_count = output.count("\n")
|
||||
position_of_last_newline = output.rindex(RUBY_VERSION >= '1.9' ? /\n/ : ?\n)
|
||||
if position_of_last_newline
|
||||
after_last_newline = output[position_of_last_newline + 1 .. -1]
|
||||
ends_with_newline = after_last_newline[/\A(?:<\/span>)*\z/]
|
||||
line_count += 1 if not ends_with_newline
|
||||
end
|
||||
|
||||
case mode
|
||||
when :inline
|
||||
max_width = (start + line_count).to_s.size
|
||||
line_number = start
|
||||
output.gsub!(/^.*$\n?/) do |line|
|
||||
line_number_text = bolding.call line_number
|
||||
indent = ' ' * (max_width - line_number.to_s.size) # TODO: Optimize (10^x)
|
||||
line_number += 1
|
||||
"<span class=\"line-numbers\">#{indent}#{line_number_text}</span>#{line}"
|
||||
end
|
||||
|
||||
when :table
|
||||
line_numbers = (start ... start + line_count).map(&bolding).join("\n")
|
||||
line_numbers << "\n"
|
||||
line_numbers_table_template = Output::TABLE.apply('LINE_NUMBERS', line_numbers)
|
||||
|
||||
output.gsub!(/<\/div>\n/, '</div>')
|
||||
output.wrap_in! line_numbers_table_template
|
||||
output.wrapped_in = :div
|
||||
|
||||
when :list
|
||||
raise NotImplementedError, 'The :list option is no longer available. Use :table.'
|
||||
|
||||
else
|
||||
raise ArgumentError, 'Unknown value %p for mode: expected one of %p' %
|
||||
[mode, [:table, :inline]]
|
||||
end
|
||||
|
||||
output
|
||||
end
|
||||
|
||||
end
|
||||
|
||||
end
|
||||
|
||||
end
|
||||
end
|
||||
|
|
@ -0,0 +1,166 @@
|
|||
module CodeRay
|
||||
module Encoders
|
||||
|
||||
class HTML
|
||||
|
||||
# This module is included in the output String of the HTML Encoder.
|
||||
#
|
||||
# It provides methods like wrap, div, page etc.
|
||||
#
|
||||
# Remember to use #clone instead of #dup to keep the modules the object was
|
||||
# extended with.
|
||||
#
|
||||
# TODO: Rewrite this without monkey patching.
|
||||
module Output
|
||||
|
||||
attr_accessor :css
|
||||
|
||||
class << self
|
||||
|
||||
# Raises an exception if an object that doesn't respond to to_str is extended by Output,
|
||||
# to prevent users from misuse. Use Module#remove_method to disable.
|
||||
def extended o # :nodoc:
|
||||
warn "The Output module is intended to extend instances of String, not #{o.class}." unless o.respond_to? :to_str
|
||||
end
|
||||
|
||||
def make_stylesheet css, in_tag = false # :nodoc:
|
||||
sheet = css.stylesheet
|
||||
sheet = <<-'CSS' if in_tag
|
||||
<style type="text/css">
|
||||
#{sheet}
|
||||
</style>
|
||||
CSS
|
||||
sheet
|
||||
end
|
||||
|
||||
def page_template_for_css css # :nodoc:
|
||||
sheet = make_stylesheet css
|
||||
PAGE.apply 'CSS', sheet
|
||||
end
|
||||
|
||||
end
|
||||
|
||||
def wrapped_in? element
|
||||
wrapped_in == element
|
||||
end
|
||||
|
||||
def wrapped_in
|
||||
@wrapped_in ||= nil
|
||||
end
|
||||
attr_writer :wrapped_in
|
||||
|
||||
def wrap_in! template
|
||||
Template.wrap! self, template, 'CONTENT'
|
||||
self
|
||||
end
|
||||
|
||||
def apply_title! title
|
||||
self.sub!(/(<title>)(<\/title>)/) { $1 + title + $2 }
|
||||
self
|
||||
end
|
||||
|
||||
def wrap! element, *args
|
||||
return self if not element or element == wrapped_in
|
||||
case element
|
||||
when :div
|
||||
raise "Can't wrap %p in %p" % [wrapped_in, element] unless wrapped_in? nil
|
||||
wrap_in! DIV
|
||||
when :span
|
||||
raise "Can't wrap %p in %p" % [wrapped_in, element] unless wrapped_in? nil
|
||||
wrap_in! SPAN
|
||||
when :page
|
||||
wrap! :div if wrapped_in? nil
|
||||
raise "Can't wrap %p in %p" % [wrapped_in, element] unless wrapped_in? :div
|
||||
wrap_in! Output.page_template_for_css(@css)
|
||||
if args.first.is_a?(Hash) && title = args.first[:title]
|
||||
apply_title! title
|
||||
end
|
||||
self
|
||||
when nil
|
||||
return self
|
||||
else
|
||||
raise "Unknown value %p for :wrap" % element
|
||||
end
|
||||
@wrapped_in = element
|
||||
self
|
||||
end
|
||||
|
||||
def stylesheet in_tag = false
|
||||
Output.make_stylesheet @css, in_tag
|
||||
end
|
||||
|
||||
#-- don't include the templates in docu
|
||||
|
||||
class Template < String # :nodoc:
|
||||
|
||||
def self.wrap! str, template, target
|
||||
target = Regexp.new(Regexp.escape("<%#{target}%>"))
|
||||
if template =~ target
|
||||
str[0,0] = $`
|
||||
str << $'
|
||||
else
|
||||
raise "Template target <%%%p%%> not found" % target
|
||||
end
|
||||
end
|
||||
|
||||
def apply target, replacement
|
||||
target = Regexp.new(Regexp.escape("<%#{target}%>"))
|
||||
if self =~ target
|
||||
Template.new($` + replacement + $')
|
||||
else
|
||||
raise "Template target <%%%p%%> not found" % target
|
||||
end
|
||||
end
|
||||
|
||||
end
|
||||
|
||||
SPAN = Template.new '<span class="CodeRay"><%CONTENT%></span>'
|
||||
|
||||
DIV = Template.new <<-DIV
|
||||
<div class="CodeRay">
|
||||
<div class="code"><pre><%CONTENT%></pre></div>
|
||||
</div>
|
||||
DIV
|
||||
|
||||
TABLE = Template.new <<-TABLE
|
||||
<table class="CodeRay"><tr>
|
||||
<td class="line-numbers" title="double click to toggle" ondblclick="with (this.firstChild.style) { display = (display == '') ? 'none' : '' }"><pre><%LINE_NUMBERS%></pre></td>
|
||||
<td class="code"><pre><%CONTENT%></pre></td>
|
||||
</tr></table>
|
||||
TABLE
|
||||
|
||||
PAGE = Template.new <<-PAGE
|
||||
<!DOCTYPE html>
|
||||
<html>
|
||||
<head>
|
||||
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
|
||||
<title></title>
|
||||
<style type="text/css">
|
||||
.CodeRay .line-numbers a {
|
||||
text-decoration: inherit;
|
||||
color: inherit;
|
||||
}
|
||||
body {
|
||||
background-color: white;
|
||||
padding: 0;
|
||||
margin: 0;
|
||||
}
|
||||
<%CSS%>
|
||||
.CodeRay {
|
||||
border: none;
|
||||
}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
|
||||
<%CONTENT%>
|
||||
</body>
|
||||
</html>
|
||||
PAGE
|
||||
|
||||
end
|
||||
|
||||
end
|
||||
|
||||
end
|
||||
end
|
||||
|
|
@ -0,0 +1,83 @@
|
|||
module CodeRay
|
||||
module Encoders
|
||||
|
||||
# A simple JSON Encoder.
|
||||
#
|
||||
# Example:
|
||||
# CodeRay.scan('puts "Hello world!"', :ruby).json
|
||||
# yields
|
||||
# [
|
||||
# {"type"=>"text", "text"=>"puts", "kind"=>"ident"},
|
||||
# {"type"=>"text", "text"=>" ", "kind"=>"space"},
|
||||
# {"type"=>"block", "action"=>"open", "kind"=>"string"},
|
||||
# {"type"=>"text", "text"=>"\"", "kind"=>"delimiter"},
|
||||
# {"type"=>"text", "text"=>"Hello world!", "kind"=>"content"},
|
||||
# {"type"=>"text", "text"=>"\"", "kind"=>"delimiter"},
|
||||
# {"type"=>"block", "action"=>"close", "kind"=>"string"},
|
||||
# ]
|
||||
class JSON < Encoder
|
||||
|
||||
begin
|
||||
require 'json'
|
||||
rescue LoadError
|
||||
begin
|
||||
require 'rubygems' unless defined? Gem
|
||||
gem 'json'
|
||||
require 'json'
|
||||
rescue LoadError
|
||||
$stderr.puts "The JSON encoder needs the JSON library.\n" \
|
||||
"Please gem install json."
|
||||
raise
|
||||
end
|
||||
end
|
||||
|
||||
register_for :json
|
||||
FILE_EXTENSION = 'json'
|
||||
|
||||
protected
|
||||
def setup options
|
||||
super
|
||||
|
||||
@first = true
|
||||
@out << '['
|
||||
end
|
||||
|
||||
def finish options
|
||||
@out << ']'
|
||||
end
|
||||
|
||||
def append data
|
||||
if @first
|
||||
@first = false
|
||||
else
|
||||
@out << ','
|
||||
end
|
||||
|
||||
@out << data.to_json
|
||||
end
|
||||
|
||||
public
|
||||
def text_token text, kind
|
||||
append :type => 'text', :text => text, :kind => kind
|
||||
end
|
||||
|
||||
def begin_group kind
|
||||
append :type => 'block', :action => 'open', :kind => kind
|
||||
end
|
||||
|
||||
def end_group kind
|
||||
append :type => 'block', :action => 'close', :kind => kind
|
||||
end
|
||||
|
||||
def begin_line kind
|
||||
append :type => 'block', :action => 'begin_line', :kind => kind
|
||||
end
|
||||
|
||||
def end_line kind
|
||||
append :type => 'block', :action => 'end_line', :kind => kind
|
||||
end
|
||||
|
||||
end
|
||||
|
||||
end
|
||||
end
|
||||
|
|
@ -0,0 +1,45 @@
|
|||
module CodeRay
|
||||
module Encoders
|
||||
|
||||
# Counts the LoC (Lines of Code). Returns an Integer >= 0.
|
||||
#
|
||||
# Alias: +loc+
|
||||
#
|
||||
# Everything that is not comment, markup, doctype/shebang, or an empty line,
|
||||
# is considered to be code.
|
||||
#
|
||||
# For example,
|
||||
# * HTML files not containing JavaScript have 0 LoC
|
||||
# * in a Java class without comments, LoC is the number of non-empty lines
|
||||
#
|
||||
# A Scanner class should define the token kinds that are not code in the
|
||||
# KINDS_NOT_LOC constant, which defaults to [:comment, :doctype].
|
||||
class LinesOfCode < TokenKindFilter
|
||||
|
||||
register_for :lines_of_code
|
||||
|
||||
NON_EMPTY_LINE = /^\s*\S.*$/
|
||||
|
||||
protected
|
||||
|
||||
def setup options
|
||||
if scanner
|
||||
kinds_not_loc = scanner.class::KINDS_NOT_LOC
|
||||
else
|
||||
warn "Tokens have no associated scanner, counting all nonempty lines." if $VERBOSE
|
||||
kinds_not_loc = CodeRay::Scanners::Scanner::KINDS_NOT_LOC
|
||||
end
|
||||
|
||||
options[:exclude] = kinds_not_loc
|
||||
|
||||
super options
|
||||
end
|
||||
|
||||
def finish options
|
||||
output @tokens.text.scan(NON_EMPTY_LINE).size
|
||||
end
|
||||
|
||||
end
|
||||
|
||||
end
|
||||
end
|
||||
|
|
@ -0,0 +1,18 @@
|
|||
module CodeRay
|
||||
module Encoders
|
||||
|
||||
# = Null Encoder
|
||||
#
|
||||
# Does nothing and returns an empty string.
|
||||
class Null < Encoder
|
||||
|
||||
register_for :null
|
||||
|
||||
def text_token text, kind
|
||||
# do nothing
|
||||
end
|
||||
|
||||
end
|
||||
|
||||
end
|
||||
end
|
||||
|
|
@ -0,0 +1,24 @@
|
|||
module CodeRay
|
||||
module Encoders
|
||||
|
||||
load :html
|
||||
|
||||
# Wraps the output into a HTML page, using CSS classes and
|
||||
# line numbers in the table format by default.
|
||||
#
|
||||
# See Encoders::HTML for available options.
|
||||
class Page < HTML
|
||||
|
||||
FILE_EXTENSION = 'html'
|
||||
|
||||
register_for :page
|
||||
|
||||
DEFAULT_OPTIONS = HTML::DEFAULT_OPTIONS.merge \
|
||||
:css => :class,
|
||||
:wrap => :page,
|
||||
:line_numbers => :table
|
||||
|
||||
end
|
||||
|
||||
end
|
||||
end
|
||||
|
|
@ -0,0 +1,23 @@
|
|||
module CodeRay
|
||||
module Encoders
|
||||
|
||||
load :html
|
||||
|
||||
# Wraps HTML output into a SPAN element, using inline styles by default.
|
||||
#
|
||||
# See Encoders::HTML for available options.
|
||||
class Span < HTML
|
||||
|
||||
FILE_EXTENSION = 'span.html'
|
||||
|
||||
register_for :span
|
||||
|
||||
DEFAULT_OPTIONS = HTML::DEFAULT_OPTIONS.merge \
|
||||
:css => :style,
|
||||
:wrap => :span,
|
||||
:line_numbers => false
|
||||
|
||||
end
|
||||
|
||||
end
|
||||
end
|
||||
|
|
@ -0,0 +1,96 @@
|
|||
module CodeRay
|
||||
module Encoders
|
||||
|
||||
# Makes a statistic for the given tokens.
|
||||
#
|
||||
# Alias: +stats+
|
||||
class Statistic < Encoder
|
||||
|
||||
register_for :statistic
|
||||
|
||||
attr_reader :type_stats, :real_token_count # :nodoc:
|
||||
|
||||
TypeStats = Struct.new :count, :size # :nodoc:
|
||||
|
||||
protected
|
||||
|
||||
def setup options
|
||||
super
|
||||
|
||||
@type_stats = Hash.new { |h, k| h[k] = TypeStats.new 0, 0 }
|
||||
@real_token_count = 0
|
||||
end
|
||||
|
||||
STATS = <<-STATS # :nodoc:
|
||||
|
||||
Code Statistics
|
||||
|
||||
Tokens %8d
|
||||
Non-Whitespace %8d
|
||||
Bytes Total %8d
|
||||
|
||||
Token Types (%d):
|
||||
type count ratio size (average)
|
||||
-------------------------------------------------------------
|
||||
%s
|
||||
STATS
|
||||
|
||||
TOKEN_TYPES_ROW = <<-TKR # :nodoc:
|
||||
%-20s %8d %6.2f %% %5.1f
|
||||
TKR
|
||||
|
||||
def finish options
|
||||
all = @type_stats['TOTAL']
|
||||
all_count, all_size = all.count, all.size
|
||||
@type_stats.each do |type, stat|
|
||||
stat.size /= stat.count.to_f
|
||||
end
|
||||
types_stats = @type_stats.sort_by { |k, v| [-v.count, k.to_s] }.map do |k, v|
|
||||
TOKEN_TYPES_ROW % [k, v.count, 100.0 * v.count / all_count, v.size]
|
||||
end.join
|
||||
@out << STATS % [
|
||||
all_count, @real_token_count, all_size,
|
||||
@type_stats.delete_if { |k, v| k.is_a? String }.size,
|
||||
types_stats
|
||||
]
|
||||
|
||||
super
|
||||
end
|
||||
|
||||
public
|
||||
|
||||
def text_token text, kind
|
||||
@real_token_count += 1 unless kind == :space
|
||||
@type_stats[kind].count += 1
|
||||
@type_stats[kind].size += text.size
|
||||
@type_stats['TOTAL'].size += text.size
|
||||
@type_stats['TOTAL'].count += 1
|
||||
end
|
||||
|
||||
# TODO Hierarchy handling
|
||||
def begin_group kind
|
||||
block_token ':begin_group', kind
|
||||
end
|
||||
|
||||
def end_group kind
|
||||
block_token ':end_group', kind
|
||||
end
|
||||
|
||||
def begin_line kind
|
||||
block_token ':begin_line', kind
|
||||
end
|
||||
|
||||
def end_line kind
|
||||
block_token ':end_line', kind
|
||||
end
|
||||
|
||||
def block_token action, kind
|
||||
@type_stats['TOTAL'].count += 1
|
||||
@type_stats[action].count += 1
|
||||
@type_stats[kind].count += 1
|
||||
end
|
||||
|
||||
end
|
||||
|
||||
end
|
||||
end
|
||||
|
|
@ -0,0 +1,179 @@
|
|||
module CodeRay
|
||||
module Encoders
|
||||
|
||||
# Outputs code highlighted for a color terminal.
|
||||
#
|
||||
# Note: This encoder is in beta. It currently doesn't use the Styles.
|
||||
#
|
||||
# Alias: +term+
|
||||
#
|
||||
# == Authors & License
|
||||
#
|
||||
# By Rob Aldred (http://robaldred.co.uk)
|
||||
#
|
||||
# Based on idea by Nathan Weizenbaum (http://nex-3.com)
|
||||
#
|
||||
# MIT License (http://www.opensource.org/licenses/mit-license.php)
|
||||
class Terminal < Encoder
|
||||
|
||||
register_for :terminal
|
||||
|
||||
TOKEN_COLORS = {
|
||||
:annotation => '35',
|
||||
:attribute_name => '33',
|
||||
:attribute_value => '31',
|
||||
:binary => '1;35',
|
||||
:char => {
|
||||
:self => '36', :delimiter => '1;34'
|
||||
},
|
||||
:class => '1;35',
|
||||
:class_variable => '36',
|
||||
:color => '32',
|
||||
:comment => '37',
|
||||
:complex => '1;34',
|
||||
:constant => ['1;34', '4'],
|
||||
:decoration => '35',
|
||||
:definition => '1;32',
|
||||
:directive => ['32', '4'],
|
||||
:doc => '46',
|
||||
:doctype => '1;30',
|
||||
:doc_string => ['31', '4'],
|
||||
:entity => '33',
|
||||
:error => ['1;33', '41'],
|
||||
:exception => '1;31',
|
||||
:float => '1;35',
|
||||
:function => '1;34',
|
||||
:global_variable => '42',
|
||||
:hex => '1;36',
|
||||
:include => '33',
|
||||
:integer => '1;34',
|
||||
:key => '35',
|
||||
:label => '1;15',
|
||||
:local_variable => '33',
|
||||
:octal => '1;35',
|
||||
:operator_name => '1;29',
|
||||
:predefined_constant => '1;36',
|
||||
:predefined_type => '1;30',
|
||||
:predefined => ['4', '1;34'],
|
||||
:preprocessor => '36',
|
||||
:pseudo_class => '1;34',
|
||||
:regexp => {
|
||||
:self => '31',
|
||||
:content => '31',
|
||||
:delimiter => '1;29',
|
||||
:modifier => '35',
|
||||
:function => '1;29'
|
||||
},
|
||||
:reserved => '1;31',
|
||||
:shell => {
|
||||
:self => '42',
|
||||
:content => '1;29',
|
||||
:delimiter => '37',
|
||||
},
|
||||
:string => {
|
||||
:self => '32',
|
||||
:modifier => '1;32',
|
||||
:escape => '1;36',
|
||||
:delimiter => '1;32',
|
||||
},
|
||||
:symbol => '1;32',
|
||||
:tag => '1;34',
|
||||
:type => '1;34',
|
||||
:value => '36',
|
||||
:variable => '1;34',
|
||||
|
||||
:insert => '42',
|
||||
:delete => '41',
|
||||
:change => '44',
|
||||
:head => '45'
|
||||
}
|
||||
TOKEN_COLORS[:keyword] = TOKEN_COLORS[:reserved]
|
||||
TOKEN_COLORS[:method] = TOKEN_COLORS[:function]
|
||||
TOKEN_COLORS[:imaginary] = TOKEN_COLORS[:complex]
|
||||
TOKEN_COLORS[:begin_group] = TOKEN_COLORS[:end_group] =
|
||||
TOKEN_COLORS[:escape] = TOKEN_COLORS[:delimiter]
|
||||
|
||||
protected
|
||||
|
||||
def setup(options)
|
||||
super
|
||||
@opened = []
|
||||
@subcolors = nil
|
||||
end
|
||||
|
||||
public
|
||||
|
||||
def text_token text, kind
|
||||
if color = (@subcolors || TOKEN_COLORS)[kind]
|
||||
if Hash === color
|
||||
if color[:self]
|
||||
color = color[:self]
|
||||
else
|
||||
@out << text
|
||||
return
|
||||
end
|
||||
end
|
||||
|
||||
@out << ansi_colorize(color)
|
||||
@out << text.gsub("\n", ansi_clear + "\n" + ansi_colorize(color))
|
||||
@out << ansi_clear
|
||||
@out << ansi_colorize(@subcolors[:self]) if @subcolors && @subcolors[:self]
|
||||
else
|
||||
@out << text
|
||||
end
|
||||
end
|
||||
|
||||
def begin_group kind
|
||||
@opened << kind
|
||||
@out << open_token(kind)
|
||||
end
|
||||
alias begin_line begin_group
|
||||
|
||||
def end_group kind
|
||||
if @opened.empty?
|
||||
# nothing to close
|
||||
else
|
||||
@opened.pop
|
||||
@out << ansi_clear
|
||||
@out << open_token(@opened.last)
|
||||
end
|
||||
end
|
||||
|
||||
def end_line kind
|
||||
if @opened.empty?
|
||||
# nothing to close
|
||||
else
|
||||
@opened.pop
|
||||
# whole lines to be highlighted,
|
||||
# eg. added/modified/deleted lines in a diff
|
||||
@out << "\t" * 100 + ansi_clear
|
||||
@out << open_token(@opened.last)
|
||||
end
|
||||
end
|
||||
|
||||
private
|
||||
|
||||
def open_token kind
|
||||
if color = TOKEN_COLORS[kind]
|
||||
if Hash === color
|
||||
@subcolors = color
|
||||
ansi_colorize(color[:self]) if color[:self]
|
||||
else
|
||||
@subcolors = {}
|
||||
ansi_colorize(color)
|
||||
end
|
||||
else
|
||||
@subcolors = nil
|
||||
''
|
||||
end
|
||||
end
|
||||
|
||||
def ansi_colorize(color)
|
||||
Array(color).map { |c| "\e[#{c}m" }.join
|
||||
end
|
||||
def ansi_clear
|
||||
ansi_colorize(0)
|
||||
end
|
||||
end
|
||||
end
|
||||
end
|
||||
|
|
@ -0,0 +1,46 @@
|
|||
module CodeRay
|
||||
module Encoders
|
||||
|
||||
# Concats the tokens into a single string, resulting in the original
|
||||
# code string if no tokens were removed.
|
||||
#
|
||||
# Alias: +plain+, +plaintext+
|
||||
#
|
||||
# == Options
|
||||
#
|
||||
# === :separator
|
||||
# A separator string to join the tokens.
|
||||
#
|
||||
# Default: empty String
|
||||
class Text < Encoder
|
||||
|
||||
register_for :text
|
||||
|
||||
FILE_EXTENSION = 'txt'
|
||||
|
||||
DEFAULT_OPTIONS = {
|
||||
:separator => nil
|
||||
}
|
||||
|
||||
def text_token text, kind
|
||||
super
|
||||
|
||||
if @first
|
||||
@first = false
|
||||
else
|
||||
@out << @sep
|
||||
end if @sep
|
||||
end
|
||||
|
||||
protected
|
||||
def setup options
|
||||
super
|
||||
|
||||
@first = true
|
||||
@sep = options[:separator]
|
||||
end
|
||||
|
||||
end
|
||||
|
||||
end
|
||||
end
|
||||
|
|
@ -0,0 +1,111 @@
|
|||
module CodeRay
|
||||
module Encoders
|
||||
|
||||
load :filter
|
||||
|
||||
# A Filter that selects tokens based on their token kind.
|
||||
#
|
||||
# == Options
|
||||
#
|
||||
# === :exclude
|
||||
#
|
||||
# One or many symbols (in an Array) which shall be excluded.
|
||||
#
|
||||
# Default: []
|
||||
#
|
||||
# === :include
|
||||
#
|
||||
# One or many symbols (in an array) which shall be included.
|
||||
#
|
||||
# Default: :all, which means all tokens are included.
|
||||
#
|
||||
# Exclusion wins over inclusion.
|
||||
#
|
||||
# See also: CommentFilter
|
||||
class TokenKindFilter < Filter
|
||||
|
||||
register_for :token_kind_filter
|
||||
|
||||
DEFAULT_OPTIONS = {
|
||||
:exclude => [],
|
||||
:include => :all
|
||||
}
|
||||
|
||||
protected
|
||||
def setup options
|
||||
super
|
||||
|
||||
@group_excluded = false
|
||||
@exclude = options[:exclude]
|
||||
@exclude = Array(@exclude) unless @exclude == :all
|
||||
@include = options[:include]
|
||||
@include = Array(@include) unless @include == :all
|
||||
end
|
||||
|
||||
def include_text_token? text, kind
|
||||
include_group? kind
|
||||
end
|
||||
|
||||
def include_group? kind
|
||||
(@include == :all || @include.include?(kind)) &&
|
||||
!(@exclude == :all || @exclude.include?(kind))
|
||||
end
|
||||
|
||||
public
|
||||
|
||||
# Add the token to the output stream if +kind+ matches the conditions.
|
||||
def text_token text, kind
|
||||
super if !@group_excluded && include_text_token?(text, kind)
|
||||
end
|
||||
|
||||
# Add the token group to the output stream if +kind+ matches the
|
||||
# conditions.
|
||||
#
|
||||
# If it does not, all tokens inside the group are excluded from the
|
||||
# stream, even if their kinds match.
|
||||
def begin_group kind
|
||||
if @group_excluded
|
||||
@group_excluded += 1
|
||||
elsif include_group? kind
|
||||
super
|
||||
else
|
||||
@group_excluded = 1
|
||||
end
|
||||
end
|
||||
|
||||
# See +begin_group+.
|
||||
def begin_line kind
|
||||
if @group_excluded
|
||||
@group_excluded += 1
|
||||
elsif include_group? kind
|
||||
super
|
||||
else
|
||||
@group_excluded = 1
|
||||
end
|
||||
end
|
||||
|
||||
# Take care of re-enabling the delegation of tokens to the output stream
|
||||
# if an exluded group has ended.
|
||||
def end_group kind
|
||||
if @group_excluded
|
||||
@group_excluded -= 1
|
||||
@group_excluded = false if @group_excluded.zero?
|
||||
else
|
||||
super
|
||||
end
|
||||
end
|
||||
|
||||
# See +end_group+.
|
||||
def end_line kind
|
||||
if @group_excluded
|
||||
@group_excluded -= 1
|
||||
@group_excluded = false if @group_excluded.zero?
|
||||
else
|
||||
super
|
||||
end
|
||||
end
|
||||
|
||||
end
|
||||
|
||||
end
|
||||
end
|
||||
|
|
@ -0,0 +1,72 @@
|
|||
module CodeRay
|
||||
module Encoders
|
||||
|
||||
# = XML Encoder
|
||||
#
|
||||
# Uses REXML. Very slow.
|
||||
class XML < Encoder
|
||||
|
||||
register_for :xml
|
||||
|
||||
FILE_EXTENSION = 'xml'
|
||||
|
||||
autoload :REXML, 'rexml/document'
|
||||
|
||||
DEFAULT_OPTIONS = {
|
||||
:tab_width => 8,
|
||||
:pretty => -1,
|
||||
:transitive => false,
|
||||
}
|
||||
|
||||
protected
|
||||
def setup options
|
||||
super
|
||||
|
||||
@doc = REXML::Document.new
|
||||
@doc << REXML::XMLDecl.new
|
||||
@tab_width = options[:tab_width]
|
||||
@root = @node = @doc.add_element('coderay-tokens')
|
||||
end
|
||||
|
||||
def finish options
|
||||
@doc.write @out, options[:pretty], options[:transitive], true
|
||||
|
||||
super
|
||||
end
|
||||
|
||||
public
|
||||
def text_token text, kind
|
||||
if kind == :space
|
||||
token = @node
|
||||
else
|
||||
token = @node.add_element kind.to_s
|
||||
end
|
||||
text.scan(/(\x20+)|(\t+)|(\n)|[^\x20\t\n]+/) do |space, tab, nl|
|
||||
case
|
||||
when space
|
||||
token << REXML::Text.new(space, true)
|
||||
when tab
|
||||
token << REXML::Text.new(tab, true)
|
||||
when nl
|
||||
token << REXML::Text.new(nl, true)
|
||||
else
|
||||
token << REXML::Text.new($&)
|
||||
end
|
||||
end
|
||||
end
|
||||
|
||||
def begin_group kind
|
||||
@node = @node.add_element kind.to_s
|
||||
end
|
||||
|
||||
def end_group kind
|
||||
if @node == @root
|
||||
raise 'no token to close!'
|
||||
end
|
||||
@node = @node.parent
|
||||
end
|
||||
|
||||
end
|
||||
|
||||
end
|
||||
end
|
||||
|
|
@ -0,0 +1,50 @@
|
|||
autoload :YAML, 'yaml'
|
||||
|
||||
module CodeRay
|
||||
module Encoders
|
||||
|
||||
# = YAML Encoder
|
||||
#
|
||||
# Slow.
|
||||
class YAML < Encoder
|
||||
|
||||
register_for :yaml
|
||||
|
||||
FILE_EXTENSION = 'yaml'
|
||||
|
||||
protected
|
||||
def setup options
|
||||
super
|
||||
|
||||
@data = []
|
||||
end
|
||||
|
||||
def finish options
|
||||
output ::YAML.dump(@data)
|
||||
end
|
||||
|
||||
public
|
||||
def text_token text, kind
|
||||
@data << [text, kind]
|
||||
end
|
||||
|
||||
def begin_group kind
|
||||
@data << [:begin_group, kind]
|
||||
end
|
||||
|
||||
def end_group kind
|
||||
@data << [:end_group, kind]
|
||||
end
|
||||
|
||||
def begin_line kind
|
||||
@data << [:begin_line, kind]
|
||||
end
|
||||
|
||||
def end_line kind
|
||||
@data << [:end_line, kind]
|
||||
end
|
||||
|
||||
end
|
||||
|
||||
end
|
||||
end
|
||||
|
|
@ -0,0 +1,95 @@
|
|||
module CodeRay
|
||||
|
||||
# A little hack to enable CodeRay highlighting in RedCloth.
|
||||
#
|
||||
# Usage:
|
||||
# require 'coderay'
|
||||
# require 'coderay/for_redcloth'
|
||||
# RedCloth.new('@[ruby]puts "Hello, World!"@').to_html
|
||||
#
|
||||
# Make sure you have RedCloth 4.0.3 activated, for example by calling
|
||||
# require 'rubygems'
|
||||
# before RedCloth is loaded and before calling CodeRay.for_redcloth.
|
||||
module ForRedCloth
|
||||
|
||||
def self.install
|
||||
gem 'RedCloth', '>= 4.0.3' if defined? gem
|
||||
require 'redcloth'
|
||||
unless RedCloth::VERSION.to_s >= '4.0.3'
|
||||
if defined? gem
|
||||
raise 'CodeRay.for_redcloth needs RedCloth version 4.0.3 or later. ' +
|
||||
"You have #{RedCloth::VERSION}. Please gem install RedCloth."
|
||||
else
|
||||
$".delete 'redcloth.rb' # sorry, but it works
|
||||
require 'rubygems'
|
||||
return install # retry
|
||||
end
|
||||
end
|
||||
unless RedCloth::VERSION.to_s >= '4.2.2'
|
||||
warn 'CodeRay.for_redcloth works best with RedCloth version 4.2.2 or later.'
|
||||
end
|
||||
RedCloth::TextileDoc.send :include, ForRedCloth::TextileDoc
|
||||
RedCloth::Formatters::HTML.module_eval do
|
||||
def unescape(html) # :nodoc:
|
||||
replacements = {
|
||||
'&' => '&',
|
||||
'"' => '"',
|
||||
'>' => '>',
|
||||
'<' => '<',
|
||||
}
|
||||
html.gsub(/&(?:amp|quot|[gl]t);/) { |entity| replacements[entity] }
|
||||
end
|
||||
undef code, bc_open, bc_close, escape_pre
|
||||
def code(opts) # :nodoc:
|
||||
opts[:block] = true
|
||||
if !opts[:lang] && RedCloth::VERSION.to_s >= '4.2.0'
|
||||
# simulating pre-4.2 behavior
|
||||
if opts[:text].sub!(/\A\[(\w+)\]/, '')
|
||||
if CodeRay::Scanners[$1].lang == :text
|
||||
opts[:text] = $& + opts[:text]
|
||||
else
|
||||
opts[:lang] = $1
|
||||
end
|
||||
end
|
||||
end
|
||||
if opts[:lang] && !filter_coderay
|
||||
require 'coderay'
|
||||
@in_bc ||= nil
|
||||
format = @in_bc ? :div : :span
|
||||
opts[:text] = unescape(opts[:text]) unless @in_bc
|
||||
highlighted_code = CodeRay.encode opts[:text], opts[:lang], format
|
||||
highlighted_code.sub!(/\A<(span|div)/) { |m| m + pba(@in_bc || opts) }
|
||||
highlighted_code
|
||||
else
|
||||
"<code#{pba(opts)}>#{opts[:text]}</code>"
|
||||
end
|
||||
end
|
||||
def bc_open(opts) # :nodoc:
|
||||
opts[:block] = true
|
||||
@in_bc = opts
|
||||
opts[:lang] ? '' : "<pre#{pba(opts)}>"
|
||||
end
|
||||
def bc_close(opts) # :nodoc:
|
||||
opts = @in_bc
|
||||
@in_bc = nil
|
||||
opts[:lang] ? '' : "</pre>\n"
|
||||
end
|
||||
def escape_pre(text) # :nodoc:
|
||||
if @in_bc ||= nil
|
||||
text
|
||||
else
|
||||
html_esc(text, :html_escape_preformatted)
|
||||
end
|
||||
end
|
||||
end
|
||||
end
|
||||
|
||||
module TextileDoc # :nodoc:
|
||||
attr_accessor :filter_coderay
|
||||
end
|
||||
|
||||
end
|
||||
|
||||
end
|
||||
|
||||
CodeRay::ForRedCloth.install
|
||||
|
|
@ -0,0 +1,143 @@
|
|||
module CodeRay
|
||||
|
||||
# = FileType
|
||||
#
|
||||
# A simple filetype recognizer.
|
||||
#
|
||||
# == Usage
|
||||
#
|
||||
# # determine the type of the given
|
||||
# lang = FileType[file_name]
|
||||
#
|
||||
# # return :text if the file type is unknown
|
||||
# lang = FileType.fetch file_name, :text
|
||||
#
|
||||
# # try the shebang line, too
|
||||
# lang = FileType.fetch file_name, :text, true
|
||||
module FileType
|
||||
|
||||
UnknownFileType = Class.new Exception
|
||||
|
||||
class << self
|
||||
|
||||
# Try to determine the file type of the file.
|
||||
#
|
||||
# +filename+ is a relative or absolute path to a file.
|
||||
#
|
||||
# The file itself is only accessed when +read_shebang+ is set to true.
|
||||
# That means you can get filetypes from files that don't exist.
|
||||
def [] filename, read_shebang = false
|
||||
name = File.basename filename
|
||||
ext = File.extname(name).sub(/^\./, '') # from last dot, delete the leading dot
|
||||
ext2 = filename.to_s[/\.(.*)/, 1] # from first dot
|
||||
|
||||
type =
|
||||
TypeFromExt[ext] ||
|
||||
TypeFromExt[ext.downcase] ||
|
||||
(TypeFromExt[ext2] if ext2) ||
|
||||
(TypeFromExt[ext2.downcase] if ext2) ||
|
||||
TypeFromName[name] ||
|
||||
TypeFromName[name.downcase]
|
||||
type ||= shebang(filename) if read_shebang
|
||||
|
||||
type
|
||||
end
|
||||
|
||||
# This works like Hash#fetch.
|
||||
#
|
||||
# If the filetype cannot be found, the +default+ value
|
||||
# is returned.
|
||||
def fetch filename, default = nil, read_shebang = false
|
||||
if default && block_given?
|
||||
warn 'Block supersedes default value argument; use either.'
|
||||
end
|
||||
|
||||
if type = self[filename, read_shebang]
|
||||
type
|
||||
else
|
||||
return yield if block_given?
|
||||
return default if default
|
||||
raise UnknownFileType, 'Could not determine type of %p.' % filename
|
||||
end
|
||||
end
|
||||
|
||||
protected
|
||||
|
||||
def shebang filename
|
||||
return unless File.exist? filename
|
||||
File.open filename, 'r' do |f|
|
||||
if first_line = f.gets
|
||||
if type = first_line[TypeFromShebang]
|
||||
type.to_sym
|
||||
end
|
||||
end
|
||||
end
|
||||
end
|
||||
|
||||
end
|
||||
|
||||
TypeFromExt = {
|
||||
'c' => :c,
|
||||
'cfc' => :xml,
|
||||
'cfm' => :xml,
|
||||
'clj' => :clojure,
|
||||
'css' => :css,
|
||||
'diff' => :diff,
|
||||
'dpr' => :delphi,
|
||||
'erb' => :erb,
|
||||
'gemspec' => :ruby,
|
||||
'groovy' => :groovy,
|
||||
'gvy' => :groovy,
|
||||
'h' => :c,
|
||||
'haml' => :haml,
|
||||
'htm' => :html,
|
||||
'html' => :html,
|
||||
'html.erb' => :erb,
|
||||
'java' => :java,
|
||||
'js' => :java_script,
|
||||
'json' => :json,
|
||||
'mab' => :ruby,
|
||||
'pas' => :delphi,
|
||||
'patch' => :diff,
|
||||
'php' => :php,
|
||||
'php3' => :php,
|
||||
'php4' => :php,
|
||||
'php5' => :php,
|
||||
'prawn' => :ruby,
|
||||
'py' => :python,
|
||||
'py3' => :python,
|
||||
'pyw' => :python,
|
||||
'rake' => :ruby,
|
||||
'raydebug' => :raydebug,
|
||||
'rb' => :ruby,
|
||||
'rbw' => :ruby,
|
||||
'rhtml' => :erb,
|
||||
'rjs' => :ruby,
|
||||
'rpdf' => :ruby,
|
||||
'ru' => :ruby,
|
||||
'rxml' => :ruby,
|
||||
# 'sch' => :scheme,
|
||||
'sql' => :sql,
|
||||
# 'ss' => :scheme,
|
||||
'tmproj' => :xml,
|
||||
'xhtml' => :html,
|
||||
'xml' => :xml,
|
||||
'yaml' => :yaml,
|
||||
'yml' => :yaml,
|
||||
}
|
||||
for cpp_alias in %w[cc cpp cp cxx c++ C hh hpp h++ cu]
|
||||
TypeFromExt[cpp_alias] = :cpp
|
||||
end
|
||||
|
||||
TypeFromShebang = /\b(?:ruby|perl|python|sh)\b/
|
||||
|
||||
TypeFromName = {
|
||||
'Capfile' => :ruby,
|
||||
'Rakefile' => :ruby,
|
||||
'Rantfile' => :ruby,
|
||||
'Gemfile' => :ruby,
|
||||
}
|
||||
|
||||
end
|
||||
|
||||
end
|
||||
|
|
@ -0,0 +1,41 @@
|
|||
module CodeRay
|
||||
|
||||
# A simplified interface to the gzip library +zlib+ (from the Ruby Standard Library.)
|
||||
module GZip
|
||||
|
||||
require 'zlib'
|
||||
|
||||
# The default zipping level. 7 zips good and fast.
|
||||
DEFAULT_GZIP_LEVEL = 7
|
||||
|
||||
# Unzips the given string +s+.
|
||||
#
|
||||
# Example:
|
||||
# require 'gzip_simple'
|
||||
# print GZip.gunzip(File.read('adresses.gz'))
|
||||
def GZip.gunzip s
|
||||
Zlib::Inflate.inflate s
|
||||
end
|
||||
|
||||
# Zips the given string +s+.
|
||||
#
|
||||
# Example:
|
||||
# require 'gzip_simple'
|
||||
# File.open('adresses.gz', 'w') do |file
|
||||
# file.write GZip.gzip('Mum: 0123 456 789', 9)
|
||||
# end
|
||||
#
|
||||
# If you provide a +level+, you can control how strong
|
||||
# the string is compressed:
|
||||
# - 0: no compression, only convert to gzip format
|
||||
# - 1: compress fast
|
||||
# - 7: compress more, but still fast (default)
|
||||
# - 8: compress more, slower
|
||||
# - 9: compress best, very slow
|
||||
def GZip.gzip s, level = DEFAULT_GZIP_LEVEL
|
||||
Zlib::Deflate.new(level).deflate s, Zlib::FINISH
|
||||
end
|
||||
|
||||
end
|
||||
|
||||
end
|
||||
|
|
@ -0,0 +1,283 @@
|
|||
module CodeRay
|
||||
|
||||
# = PluginHost
|
||||
#
|
||||
# A simple subclass/subfolder plugin system.
|
||||
#
|
||||
# Example:
|
||||
# class Generators
|
||||
# extend PluginHost
|
||||
# plugin_path 'app/generators'
|
||||
# end
|
||||
#
|
||||
# class Generator
|
||||
# extend Plugin
|
||||
# PLUGIN_HOST = Generators
|
||||
# end
|
||||
#
|
||||
# class FancyGenerator < Generator
|
||||
# register_for :fancy
|
||||
# end
|
||||
#
|
||||
# Generators[:fancy] #-> FancyGenerator
|
||||
# # or
|
||||
# CodeRay.require_plugin 'Generators/fancy'
|
||||
# # or
|
||||
# Generators::Fancy
|
||||
module PluginHost
|
||||
|
||||
# Raised if Encoders::[] fails because:
|
||||
# * a file could not be found
|
||||
# * the requested Plugin is not registered
|
||||
PluginNotFound = Class.new LoadError
|
||||
HostNotFound = Class.new LoadError
|
||||
|
||||
PLUGIN_HOSTS = []
|
||||
PLUGIN_HOSTS_BY_ID = {} # dummy hash
|
||||
|
||||
# Loads all plugins using list and load.
|
||||
def load_all
|
||||
for plugin in list
|
||||
load plugin
|
||||
end
|
||||
end
|
||||
|
||||
# Returns the Plugin for +id+.
|
||||
#
|
||||
# Example:
|
||||
# yaml_plugin = MyPluginHost[:yaml]
|
||||
def [] id, *args, &blk
|
||||
plugin = validate_id(id)
|
||||
begin
|
||||
plugin = plugin_hash.[] plugin, *args, &blk
|
||||
end while plugin.is_a? Symbol
|
||||
plugin
|
||||
end
|
||||
|
||||
alias load []
|
||||
|
||||
# Tries to +load+ the missing plugin by translating +const+ to the
|
||||
# underscore form (eg. LinesOfCode becomes lines_of_code).
|
||||
def const_missing const
|
||||
id = const.to_s.
|
||||
gsub(/([A-Z]+)([A-Z][a-z])/,'\1_\2').
|
||||
gsub(/([a-z\d])([A-Z])/,'\1_\2').
|
||||
downcase
|
||||
load id
|
||||
end
|
||||
|
||||
class << self
|
||||
|
||||
# Adds the module/class to the PLUGIN_HOSTS list.
|
||||
def extended mod
|
||||
PLUGIN_HOSTS << mod
|
||||
end
|
||||
|
||||
end
|
||||
|
||||
# The path where the plugins can be found.
|
||||
def plugin_path *args
|
||||
unless args.empty?
|
||||
@plugin_path = File.expand_path File.join(*args)
|
||||
end
|
||||
@plugin_path ||= ''
|
||||
end
|
||||
|
||||
# Map a plugin_id to another.
|
||||
#
|
||||
# Usage: Put this in a file plugin_path/_map.rb.
|
||||
#
|
||||
# class MyColorHost < PluginHost
|
||||
# map :navy => :dark_blue,
|
||||
# :maroon => :brown,
|
||||
# :luna => :moon
|
||||
# end
|
||||
def map hash
|
||||
for from, to in hash
|
||||
from = validate_id from
|
||||
to = validate_id to
|
||||
plugin_hash[from] = to unless plugin_hash.has_key? from
|
||||
end
|
||||
end
|
||||
|
||||
# Define the default plugin to use when no plugin is found
|
||||
# for a given id, or return the default plugin.
|
||||
#
|
||||
# See also map.
|
||||
#
|
||||
# class MyColorHost < PluginHost
|
||||
# map :navy => :dark_blue
|
||||
# default :gray
|
||||
# end
|
||||
#
|
||||
# MyColorHost.default # loads and returns the Gray plugin
|
||||
def default id = nil
|
||||
if id
|
||||
id = validate_id id
|
||||
raise "The default plugin can't be named \"default\"." if id == :default
|
||||
plugin_hash[:default] = id
|
||||
else
|
||||
load :default
|
||||
end
|
||||
end
|
||||
|
||||
# Every plugin must register itself for +id+ by calling register_for,
|
||||
# which calls this method.
|
||||
#
|
||||
# See Plugin#register_for.
|
||||
def register plugin, id
|
||||
plugin_hash[validate_id(id)] = plugin
|
||||
end
|
||||
|
||||
# A Hash of plugion_id => Plugin pairs.
|
||||
def plugin_hash
|
||||
@plugin_hash ||= make_plugin_hash
|
||||
end
|
||||
|
||||
# Returns an array of all .rb files in the plugin path.
|
||||
#
|
||||
# The extension .rb is not included.
|
||||
def list
|
||||
Dir[path_to('*')].select do |file|
|
||||
File.basename(file)[/^(?!_)\w+\.rb$/]
|
||||
end.map do |file|
|
||||
File.basename(file, '.rb').to_sym
|
||||
end
|
||||
end
|
||||
|
||||
# Returns an array of all Plugins.
|
||||
#
|
||||
# Note: This loads all plugins using load_all.
|
||||
def all_plugins
|
||||
load_all
|
||||
plugin_hash.values.grep(Class)
|
||||
end
|
||||
|
||||
# Loads the map file (see map).
|
||||
#
|
||||
# This is done automatically when plugin_path is called.
|
||||
def load_plugin_map
|
||||
mapfile = path_to '_map'
|
||||
@plugin_map_loaded = true
|
||||
if File.exist? mapfile
|
||||
require mapfile
|
||||
true
|
||||
else
|
||||
false
|
||||
end
|
||||
end
|
||||
|
||||
protected
|
||||
|
||||
# Return a plugin hash that automatically loads plugins.
|
||||
def make_plugin_hash
|
||||
@plugin_map_loaded ||= false
|
||||
Hash.new do |h, plugin_id|
|
||||
id = validate_id(plugin_id)
|
||||
path = path_to id
|
||||
begin
|
||||
require path
|
||||
rescue LoadError => boom
|
||||
if @plugin_map_loaded
|
||||
if h.has_key?(:default)
|
||||
warn '%p could not load plugin %p; falling back to %p' % [self, id, h[:default]]
|
||||
h[:default]
|
||||
else
|
||||
raise PluginNotFound, '%p could not load plugin %p: %s' % [self, id, boom]
|
||||
end
|
||||
else
|
||||
load_plugin_map
|
||||
h[plugin_id]
|
||||
end
|
||||
else
|
||||
# Plugin should have registered by now
|
||||
if h.has_key? id
|
||||
h[id]
|
||||
else
|
||||
raise PluginNotFound, "No #{self.name} plugin for #{id.inspect} found in #{path}."
|
||||
end
|
||||
end
|
||||
end
|
||||
end
|
||||
|
||||
# Returns the expected path to the plugin file for the given id.
|
||||
def path_to plugin_id
|
||||
File.join plugin_path, "#{plugin_id}.rb"
|
||||
end
|
||||
|
||||
# Converts +id+ to a Symbol if it is a String,
|
||||
# or returns +id+ if it already is a Symbol.
|
||||
#
|
||||
# Raises +ArgumentError+ for all other objects, or if the
|
||||
# given String includes non-alphanumeric characters (\W).
|
||||
def validate_id id
|
||||
if id.is_a? Symbol or id.nil?
|
||||
id
|
||||
elsif id.is_a? String
|
||||
if id[/\w+/] == id
|
||||
id.downcase.to_sym
|
||||
else
|
||||
raise ArgumentError, "Invalid id given: #{id}"
|
||||
end
|
||||
else
|
||||
raise ArgumentError, "String or Symbol expected, but #{id.class} given."
|
||||
end
|
||||
end
|
||||
|
||||
end
|
||||
|
||||
|
||||
# = Plugin
|
||||
#
|
||||
# Plugins have to include this module.
|
||||
#
|
||||
# IMPORTANT: Use extend for this module.
|
||||
#
|
||||
# See CodeRay::PluginHost for examples.
|
||||
module Plugin
|
||||
|
||||
attr_reader :plugin_id
|
||||
|
||||
# Register this class for the given +id+.
|
||||
#
|
||||
# Example:
|
||||
# class MyPlugin < PluginHost::BaseClass
|
||||
# register_for :my_id
|
||||
# ...
|
||||
# end
|
||||
#
|
||||
# See PluginHost.register.
|
||||
def register_for id
|
||||
@plugin_id = id
|
||||
plugin_host.register self, id
|
||||
end
|
||||
|
||||
# Returns the title of the plugin, or sets it to the
|
||||
# optional argument +title+.
|
||||
def title title = nil
|
||||
if title
|
||||
@title = title.to_s
|
||||
else
|
||||
@title ||= name[/([^:]+)$/, 1]
|
||||
end
|
||||
end
|
||||
|
||||
# The PluginHost for this Plugin class.
|
||||
def plugin_host host = nil
|
||||
if host.is_a? PluginHost
|
||||
const_set :PLUGIN_HOST, host
|
||||
end
|
||||
self::PLUGIN_HOST
|
||||
end
|
||||
|
||||
def aliases
|
||||
plugin_host.load_plugin_map
|
||||
plugin_host.plugin_hash.inject [] do |aliases, (key, _)|
|
||||
aliases << key if plugin_host[key] == self
|
||||
aliases
|
||||
end
|
||||
end
|
||||
|
||||
end
|
||||
|
||||
end
|
||||
|
|
@ -0,0 +1,72 @@
|
|||
module CodeRay
|
||||
|
||||
# = WordList
|
||||
#
|
||||
# <b>A Hash subclass designed for mapping word lists to token types.</b>
|
||||
#
|
||||
# A WordList is a Hash with some additional features.
|
||||
# It is intended to be used for keyword recognition.
|
||||
#
|
||||
# WordList is optimized to be used in Scanners,
|
||||
# typically to decide whether a given ident is a special token.
|
||||
#
|
||||
# For case insensitive words use WordList::CaseIgnoring.
|
||||
#
|
||||
# Example:
|
||||
#
|
||||
# # define word arrays
|
||||
# RESERVED_WORDS = %w[
|
||||
# asm break case continue default do else
|
||||
# ]
|
||||
#
|
||||
# PREDEFINED_TYPES = %w[
|
||||
# int long short char void
|
||||
# ]
|
||||
#
|
||||
# # make a WordList
|
||||
# IDENT_KIND = WordList.new(:ident).
|
||||
# add(RESERVED_WORDS, :reserved).
|
||||
# add(PREDEFINED_TYPES, :predefined_type)
|
||||
#
|
||||
# ...
|
||||
#
|
||||
# def scan_tokens tokens, options
|
||||
# ...
|
||||
#
|
||||
# elsif scan(/[A-Za-z_][A-Za-z_0-9]*/)
|
||||
# # use it
|
||||
# kind = IDENT_KIND[match]
|
||||
# ...
|
||||
class WordList < Hash
|
||||
|
||||
# Create a new WordList with +default+ as default value.
|
||||
def initialize default = false
|
||||
super default
|
||||
end
|
||||
|
||||
# Add words to the list and associate them with +value+.
|
||||
#
|
||||
# Returns +self+, so you can concat add calls.
|
||||
def add words, value = true
|
||||
words.each { |word| self[word] = value }
|
||||
self
|
||||
end
|
||||
|
||||
end
|
||||
|
||||
|
||||
# A CaseIgnoring WordList is like a WordList, only that
|
||||
# keys are compared case-insensitively (normalizing keys using +downcase+).
|
||||
class WordList::CaseIgnoring < WordList
|
||||
|
||||
def [] key
|
||||
super key.downcase
|
||||
end
|
||||
|
||||
def []= key, value
|
||||
super key.downcase, value
|
||||
end
|
||||
|
||||
end
|
||||
|
||||
end
|
||||
|
|
@ -0,0 +1,323 @@
|
|||
# encoding: utf-8
|
||||
require 'strscan'
|
||||
|
||||
module CodeRay
|
||||
|
||||
autoload :WordList, coderay_path('helpers', 'word_list')
|
||||
|
||||
# = Scanners
|
||||
#
|
||||
# This module holds the Scanner class and its subclasses.
|
||||
# For example, the Ruby scanner is named CodeRay::Scanners::Ruby
|
||||
# can be found in coderay/scanners/ruby.
|
||||
#
|
||||
# Scanner also provides methods and constants for the register
|
||||
# mechanism and the [] method that returns the Scanner class
|
||||
# belonging to the given lang.
|
||||
#
|
||||
# See PluginHost.
|
||||
module Scanners
|
||||
extend PluginHost
|
||||
plugin_path File.dirname(__FILE__), 'scanners'
|
||||
|
||||
|
||||
# = Scanner
|
||||
#
|
||||
# The base class for all Scanners.
|
||||
#
|
||||
# It is a subclass of Ruby's great +StringScanner+, which
|
||||
# makes it easy to access the scanning methods inside.
|
||||
#
|
||||
# It is also +Enumerable+, so you can use it like an Array of
|
||||
# Tokens:
|
||||
#
|
||||
# require 'coderay'
|
||||
#
|
||||
# c_scanner = CodeRay::Scanners[:c].new "if (*p == '{') nest++;"
|
||||
#
|
||||
# for text, kind in c_scanner
|
||||
# puts text if kind == :operator
|
||||
# end
|
||||
#
|
||||
# # prints: (*==)++;
|
||||
#
|
||||
# OK, this is a very simple example :)
|
||||
# You can also use +map+, +any?+, +find+ and even +sort_by+,
|
||||
# if you want.
|
||||
class Scanner < StringScanner
|
||||
|
||||
extend Plugin
|
||||
plugin_host Scanners
|
||||
|
||||
# Raised if a Scanner fails while scanning
|
||||
ScanError = Class.new StandardError
|
||||
|
||||
# The default options for all scanner classes.
|
||||
#
|
||||
# Define @default_options for subclasses.
|
||||
DEFAULT_OPTIONS = { }
|
||||
|
||||
KINDS_NOT_LOC = [:comment, :doctype, :docstring]
|
||||
|
||||
attr_accessor :state
|
||||
|
||||
class << self
|
||||
|
||||
# Normalizes the given code into a string with UNIX newlines, in the
|
||||
# scanner's internal encoding, with invalid and undefined charachters
|
||||
# replaced by placeholders. Always returns a new object.
|
||||
def normalize code
|
||||
# original = code
|
||||
code = code.to_s unless code.is_a? ::String
|
||||
return code if code.empty?
|
||||
|
||||
if code.respond_to? :encoding
|
||||
code = encode_with_encoding code, self.encoding
|
||||
else
|
||||
code = to_unix code
|
||||
end
|
||||
# code = code.dup if code.eql? original
|
||||
code
|
||||
end
|
||||
|
||||
# The typical filename suffix for this scanner's language.
|
||||
def file_extension extension = lang
|
||||
@file_extension ||= extension.to_s
|
||||
end
|
||||
|
||||
# The encoding used internally by this scanner.
|
||||
def encoding name = 'UTF-8'
|
||||
@encoding ||= defined?(Encoding.find) && Encoding.find(name)
|
||||
end
|
||||
|
||||
# The lang of this Scanner class, which is equal to its Plugin ID.
|
||||
def lang
|
||||
@plugin_id
|
||||
end
|
||||
|
||||
protected
|
||||
|
||||
def encode_with_encoding code, target_encoding
|
||||
if code.encoding == target_encoding
|
||||
if code.valid_encoding?
|
||||
return to_unix(code)
|
||||
else
|
||||
source_encoding = guess_encoding code
|
||||
end
|
||||
else
|
||||
source_encoding = code.encoding
|
||||
end
|
||||
# print "encode_with_encoding from #{source_encoding} to #{target_encoding}"
|
||||
code.encode target_encoding, source_encoding, :universal_newline => true, :undef => :replace, :invalid => :replace
|
||||
end
|
||||
|
||||
def to_unix code
|
||||
code.index(?\r) ? code.gsub(/\r\n?/, "\n") : code
|
||||
end
|
||||
|
||||
def guess_encoding s
|
||||
#:nocov:
|
||||
IO.popen("file -b --mime -", "w+") do |file|
|
||||
file.write s[0, 1024]
|
||||
file.close_write
|
||||
begin
|
||||
Encoding.find file.gets[/charset=([-\w]+)/, 1]
|
||||
rescue ArgumentError
|
||||
Encoding::BINARY
|
||||
end
|
||||
end
|
||||
#:nocov:
|
||||
end
|
||||
|
||||
end
|
||||
|
||||
# Create a new Scanner.
|
||||
#
|
||||
# * +code+ is the input String and is handled by the superclass
|
||||
# StringScanner.
|
||||
# * +options+ is a Hash with Symbols as keys.
|
||||
# It is merged with the default options of the class (you can
|
||||
# overwrite default options here.)
|
||||
#
|
||||
# Else, a Tokens object is used.
|
||||
def initialize code = '', options = {}
|
||||
if self.class == Scanner
|
||||
raise NotImplementedError, "I am only the basic Scanner class. I can't scan anything. :( Use my subclasses."
|
||||
end
|
||||
|
||||
@options = self.class::DEFAULT_OPTIONS.merge options
|
||||
|
||||
super self.class.normalize(code)
|
||||
|
||||
@tokens = options[:tokens] || Tokens.new
|
||||
@tokens.scanner = self if @tokens.respond_to? :scanner=
|
||||
|
||||
setup
|
||||
end
|
||||
|
||||
# Sets back the scanner. Subclasses should redefine the reset_instance
|
||||
# method instead of this one.
|
||||
def reset
|
||||
super
|
||||
reset_instance
|
||||
end
|
||||
|
||||
# Set a new string to be scanned.
|
||||
def string= code
|
||||
code = self.class.normalize(code)
|
||||
super code
|
||||
reset_instance
|
||||
end
|
||||
|
||||
# the Plugin ID for this scanner
|
||||
def lang
|
||||
self.class.lang
|
||||
end
|
||||
|
||||
# the default file extension for this scanner
|
||||
def file_extension
|
||||
self.class.file_extension
|
||||
end
|
||||
|
||||
# Scan the code and returns all tokens in a Tokens object.
|
||||
def tokenize source = nil, options = {}
|
||||
options = @options.merge(options)
|
||||
@tokens = options[:tokens] || @tokens || Tokens.new
|
||||
@tokens.scanner = self if @tokens.respond_to? :scanner=
|
||||
case source
|
||||
when Array
|
||||
self.string = self.class.normalize(source.join)
|
||||
when nil
|
||||
reset
|
||||
else
|
||||
self.string = self.class.normalize(source)
|
||||
end
|
||||
|
||||
begin
|
||||
scan_tokens @tokens, options
|
||||
rescue => e
|
||||
message = "Error in %s#scan_tokens, initial state was: %p" % [self.class, defined?(state) && state]
|
||||
raise_inspect e.message, @tokens, message, 30, e.backtrace
|
||||
end
|
||||
|
||||
@cached_tokens = @tokens
|
||||
if source.is_a? Array
|
||||
@tokens.split_into_parts(*source.map { |part| part.size })
|
||||
else
|
||||
@tokens
|
||||
end
|
||||
end
|
||||
|
||||
# Cache the result of tokenize.
|
||||
def tokens
|
||||
@cached_tokens ||= tokenize
|
||||
end
|
||||
|
||||
# Traverse the tokens.
|
||||
def each &block
|
||||
tokens.each(&block)
|
||||
end
|
||||
include Enumerable
|
||||
|
||||
# The current line position of the scanner, starting with 1.
|
||||
# See also: #column.
|
||||
#
|
||||
# Beware, this is implemented inefficiently. It should be used
|
||||
# for debugging only.
|
||||
def line pos = self.pos
|
||||
return 1 if pos <= 0
|
||||
binary_string[0...pos].count("\n") + 1
|
||||
end
|
||||
|
||||
# The current column position of the scanner, starting with 1.
|
||||
# See also: #line.
|
||||
def column pos = self.pos
|
||||
return 1 if pos <= 0
|
||||
pos - (binary_string.rindex(?\n, pos - 1) || -1)
|
||||
end
|
||||
|
||||
# The string in binary encoding.
|
||||
#
|
||||
# To be used with #pos, which is the index of the byte the scanner
|
||||
# will scan next.
|
||||
def binary_string
|
||||
@binary_string ||=
|
||||
if string.respond_to?(:bytesize) && string.bytesize != string.size
|
||||
#:nocov:
|
||||
string.dup.force_encoding('binary')
|
||||
#:nocov:
|
||||
else
|
||||
string
|
||||
end
|
||||
end
|
||||
|
||||
protected
|
||||
|
||||
# Can be implemented by subclasses to do some initialization
|
||||
# that has to be done once per instance.
|
||||
#
|
||||
# Use reset for initialization that has to be done once per
|
||||
# scan.
|
||||
def setup # :doc:
|
||||
end
|
||||
|
||||
# This is the central method, and commonly the only one a
|
||||
# subclass implements.
|
||||
#
|
||||
# Subclasses must implement this method; it must return +tokens+
|
||||
# and must only use Tokens#<< for storing scanned tokens!
|
||||
def scan_tokens tokens, options # :doc:
|
||||
raise NotImplementedError, "#{self.class}#scan_tokens not implemented."
|
||||
end
|
||||
|
||||
# Resets the scanner.
|
||||
def reset_instance
|
||||
@tokens.clear if @tokens.respond_to?(:clear) && !@options[:keep_tokens]
|
||||
@cached_tokens = nil
|
||||
@binary_string = nil if defined? @binary_string
|
||||
end
|
||||
|
||||
# Scanner error with additional status information
|
||||
def raise_inspect msg, tokens, state = self.state || 'No state given!', ambit = 30, backtrace = caller
|
||||
raise ScanError, <<-EOE % [
|
||||
|
||||
|
||||
***ERROR in %s: %s (after %d tokens)
|
||||
|
||||
tokens:
|
||||
%s
|
||||
|
||||
current line: %d column: %d pos: %d
|
||||
matched: %p state: %p
|
||||
bol? = %p, eos? = %p
|
||||
|
||||
surrounding code:
|
||||
%p ~~ %p
|
||||
|
||||
|
||||
***ERROR***
|
||||
|
||||
EOE
|
||||
File.basename(caller[0]),
|
||||
msg,
|
||||
tokens.respond_to?(:size) ? tokens.size : 0,
|
||||
tokens.respond_to?(:last) ? tokens.last(10).map { |t| t.inspect }.join("\n") : '',
|
||||
line, column, pos,
|
||||
matched, state, bol?, eos?,
|
||||
binary_string[pos - ambit, ambit],
|
||||
binary_string[pos, ambit],
|
||||
], backtrace
|
||||
end
|
||||
|
||||
# Shorthand for scan_until(/\z/).
|
||||
# This method also avoids a JRuby 1.9 mode bug.
|
||||
def scan_rest
|
||||
rest = self.rest
|
||||
terminate
|
||||
rest
|
||||
end
|
||||
|
||||
end
|
||||
|
||||
end
|
||||
end
|
||||
|
|
@ -0,0 +1,24 @@
|
|||
module CodeRay
|
||||
module Scanners
|
||||
|
||||
map \
|
||||
:'c++' => :cpp,
|
||||
:cplusplus => :cpp,
|
||||
:ecmascript => :java_script,
|
||||
:ecma_script => :java_script,
|
||||
:rhtml => :erb,
|
||||
:eruby => :erb,
|
||||
:irb => :ruby,
|
||||
:javascript => :java_script,
|
||||
:js => :java_script,
|
||||
:pascal => :delphi,
|
||||
:patch => :diff,
|
||||
:plain => :text,
|
||||
:plaintext => :text,
|
||||
:xhtml => :html,
|
||||
:yml => :yaml
|
||||
|
||||
default :text
|
||||
|
||||
end
|
||||
end
|
||||
|
|
@ -0,0 +1,189 @@
|
|||
module CodeRay
|
||||
module Scanners
|
||||
|
||||
# Scanner for C.
|
||||
class C < Scanner
|
||||
|
||||
register_for :c
|
||||
file_extension 'c'
|
||||
|
||||
KEYWORDS = [
|
||||
'asm', 'break', 'case', 'continue', 'default', 'do',
|
||||
'else', 'enum', 'for', 'goto', 'if', 'return',
|
||||
'sizeof', 'struct', 'switch', 'typedef', 'union', 'while',
|
||||
'restrict', # added in C99
|
||||
] # :nodoc:
|
||||
|
||||
PREDEFINED_TYPES = [
|
||||
'int', 'long', 'short', 'char',
|
||||
'signed', 'unsigned', 'float', 'double',
|
||||
'bool', 'complex', # added in C99
|
||||
] # :nodoc:
|
||||
|
||||
PREDEFINED_CONSTANTS = [
|
||||
'EOF', 'NULL',
|
||||
'true', 'false', # added in C99
|
||||
] # :nodoc:
|
||||
DIRECTIVES = [
|
||||
'auto', 'extern', 'register', 'static', 'void',
|
||||
'const', 'volatile', # added in C89
|
||||
'inline', # added in C99
|
||||
] # :nodoc:
|
||||
|
||||
IDENT_KIND = WordList.new(:ident).
|
||||
add(KEYWORDS, :keyword).
|
||||
add(PREDEFINED_TYPES, :predefined_type).
|
||||
add(DIRECTIVES, :directive).
|
||||
add(PREDEFINED_CONSTANTS, :predefined_constant) # :nodoc:
|
||||
|
||||
ESCAPE = / [rbfntv\n\\'"] | x[a-fA-F0-9]{1,2} | [0-7]{1,3} /x # :nodoc:
|
||||
UNICODE_ESCAPE = / u[a-fA-F0-9]{4} | U[a-fA-F0-9]{8} /x # :nodoc:
|
||||
|
||||
protected
|
||||
|
||||
def scan_tokens encoder, options
|
||||
|
||||
state = :initial
|
||||
label_expected = true
|
||||
case_expected = false
|
||||
label_expected_before_preproc_line = nil
|
||||
in_preproc_line = false
|
||||
|
||||
until eos?
|
||||
|
||||
case state
|
||||
|
||||
when :initial
|
||||
|
||||
if match = scan(/ \s+ | \\\n /x)
|
||||
if in_preproc_line && match != "\\\n" && match.index(?\n)
|
||||
in_preproc_line = false
|
||||
label_expected = label_expected_before_preproc_line
|
||||
end
|
||||
encoder.text_token match, :space
|
||||
|
||||
elsif match = scan(%r! // [^\n\\]* (?: \\. [^\n\\]* )* | /\* (?: .*? \*/ | .* ) !mx)
|
||||
encoder.text_token match, :comment
|
||||
|
||||
elsif match = scan(/ [-+*=<>?:;,!&^|()\[\]{}~%]+ | \/=? | \.(?!\d) /x)
|
||||
label_expected = match =~ /[;\{\}]/
|
||||
if case_expected
|
||||
label_expected = true if match == ':'
|
||||
case_expected = false
|
||||
end
|
||||
encoder.text_token match, :operator
|
||||
|
||||
elsif match = scan(/ [A-Za-z_][A-Za-z_0-9]* /x)
|
||||
kind = IDENT_KIND[match]
|
||||
if kind == :ident && label_expected && !in_preproc_line && scan(/:(?!:)/)
|
||||
kind = :label
|
||||
match << matched
|
||||
else
|
||||
label_expected = false
|
||||
if kind == :keyword
|
||||
case match
|
||||
when 'case', 'default'
|
||||
case_expected = true
|
||||
end
|
||||
end
|
||||
end
|
||||
encoder.text_token match, kind
|
||||
|
||||
elsif match = scan(/L?"/)
|
||||
encoder.begin_group :string
|
||||
if match[0] == ?L
|
||||
encoder.text_token 'L', :modifier
|
||||
match = '"'
|
||||
end
|
||||
encoder.text_token match, :delimiter
|
||||
state = :string
|
||||
|
||||
elsif match = scan(/ \# \s* if \s* 0 /x)
|
||||
match << scan_until(/ ^\# (?:elif|else|endif) .*? $ | \z /xm) unless eos?
|
||||
encoder.text_token match, :comment
|
||||
|
||||
elsif match = scan(/#[ \t]*(\w*)/)
|
||||
encoder.text_token match, :preprocessor
|
||||
in_preproc_line = true
|
||||
label_expected_before_preproc_line = label_expected
|
||||
state = :include_expected if self[1] == 'include'
|
||||
|
||||
elsif match = scan(/ L?' (?: [^\'\n\\] | \\ #{ESCAPE} )? '? /ox)
|
||||
label_expected = false
|
||||
encoder.text_token match, :char
|
||||
|
||||
elsif match = scan(/\$/)
|
||||
encoder.text_token match, :ident
|
||||
|
||||
elsif match = scan(/0[xX][0-9A-Fa-f]+/)
|
||||
label_expected = false
|
||||
encoder.text_token match, :hex
|
||||
|
||||
elsif match = scan(/(?:0[0-7]+)(?![89.eEfF])/)
|
||||
label_expected = false
|
||||
encoder.text_token match, :octal
|
||||
|
||||
elsif match = scan(/(?:\d+)(?![.eEfF])L?L?/)
|
||||
label_expected = false
|
||||
encoder.text_token match, :integer
|
||||
|
||||
elsif match = scan(/\d[fF]?|\d*\.\d+(?:[eE][+-]?\d+)?[fF]?|\d+[eE][+-]?\d+[fF]?/)
|
||||
label_expected = false
|
||||
encoder.text_token match, :float
|
||||
|
||||
else
|
||||
encoder.text_token getch, :error
|
||||
|
||||
end
|
||||
|
||||
when :string
|
||||
if match = scan(/[^\\\n"]+/)
|
||||
encoder.text_token match, :content
|
||||
elsif match = scan(/"/)
|
||||
encoder.text_token match, :delimiter
|
||||
encoder.end_group :string
|
||||
state = :initial
|
||||
label_expected = false
|
||||
elsif match = scan(/ \\ (?: #{ESCAPE} | #{UNICODE_ESCAPE} ) /mox)
|
||||
encoder.text_token match, :char
|
||||
elsif match = scan(/ \\ | $ /x)
|
||||
encoder.end_group :string
|
||||
encoder.text_token match, :error
|
||||
state = :initial
|
||||
label_expected = false
|
||||
else
|
||||
raise_inspect "else case \" reached; %p not handled." % peek(1), encoder
|
||||
end
|
||||
|
||||
when :include_expected
|
||||
if match = scan(/<[^>\n]+>?|"[^"\n\\]*(?:\\.[^"\n\\]*)*"?/)
|
||||
encoder.text_token match, :include
|
||||
state = :initial
|
||||
|
||||
elsif match = scan(/\s+/)
|
||||
encoder.text_token match, :space
|
||||
state = :initial if match.index ?\n
|
||||
|
||||
else
|
||||
state = :initial
|
||||
|
||||
end
|
||||
|
||||
else
|
||||
raise_inspect 'Unknown state', encoder
|
||||
|
||||
end
|
||||
|
||||
end
|
||||
|
||||
if state == :string
|
||||
encoder.end_group :string
|
||||
end
|
||||
|
||||
encoder
|
||||
end
|
||||
|
||||
end
|
||||
|
||||
end
|
||||
end
|
||||
|
|
@ -0,0 +1,217 @@
|
|||
# encoding: utf-8
|
||||
module CodeRay
|
||||
module Scanners
|
||||
|
||||
# Clojure scanner by Licenser.
|
||||
class Clojure < Scanner
|
||||
|
||||
register_for :clojure
|
||||
file_extension 'clj'
|
||||
|
||||
SPECIAL_FORMS = %w[
|
||||
def if do let quote var fn loop recur throw try catch monitor-enter monitor-exit .
|
||||
new
|
||||
] # :nodoc:
|
||||
|
||||
CORE_FORMS = %w[
|
||||
+ - -> ->> .. / * <= < = == >= > accessor aclone add-classpath add-watch
|
||||
agent agent-error agent-errors aget alength alias all-ns alter alter-meta!
|
||||
alter-var-root amap ancestors and apply areduce array-map aset aset-boolean
|
||||
aset-byte aset-char aset-double aset-float aset-int aset-long aset-short
|
||||
assert assoc assoc! assoc-in associative? atom await await-for bases bean
|
||||
bigdec bigint binding bit-and bit-and-not bit-clear bit-flip bit-not bit-or
|
||||
bit-set bit-shift-left bit-shift-right bit-test bit-xor boolean boolean-array
|
||||
booleans bound-fn bound-fn* bound? butlast byte byte-array bytes case cast char
|
||||
char-array char-escape-string char-name-string char? chars class class?
|
||||
clear-agent-errors clojure-version coll? comment commute comp comparator
|
||||
compare compare-and-set! compile complement concat cond condp conj conj!
|
||||
cons constantly construct-proxy contains? count counted? create-ns
|
||||
create-struct cycle dec decimal? declare definline defmacro defmethod defmulti
|
||||
defn defn- defonce defprotocol defrecord defstruct deftype delay delay?
|
||||
deliver denominator deref derive descendants disj disj! dissoc dissoc!
|
||||
distinct distinct? doall doc dorun doseq dosync dotimes doto double
|
||||
double-array doubles drop drop-last drop-while empty empty? ensure
|
||||
enumeration-seq error-handler error-mode eval even? every? extend
|
||||
extend-protocol extend-type extenders extends? false? ffirst file-seq
|
||||
filter find find-doc find-ns find-var first float float-array float?
|
||||
floats flush fn fn? fnext for force format future future-call future-cancel
|
||||
future-cancelled? future-done? future? gen-class gen-interface gensym get
|
||||
get-in get-method get-proxy-class get-thread-bindings get-validator hash
|
||||
hash-map hash-set identical? identity if-let if-not ifn? import in-ns
|
||||
inc init-proxy instance? int int-array integer? interleave intern
|
||||
interpose into into-array ints io! isa? iterate iterator-seq juxt key
|
||||
keys keyword keyword? last lazy-cat lazy-seq let letfn line-seq list list*
|
||||
list? load load-file load-reader load-string loaded-libs locking long
|
||||
long-array longs loop macroexpand macroexpand-1 make-array make-hierarchy
|
||||
map map? mapcat max max-key memfn memoize merge merge-with meta methods
|
||||
min min-key mod name namespace neg? newline next nfirst nil? nnext not
|
||||
not-any? not-empty not-every? not= ns ns-aliases ns-imports ns-interns
|
||||
ns-map ns-name ns-publics ns-refers ns-resolve ns-unalias ns-unmap nth
|
||||
nthnext num number? numerator object-array odd? or parents partial
|
||||
partition pcalls peek persistent! pmap pop pop! pop-thread-bindings
|
||||
pos? pr pr-str prefer-method prefers print print-namespace-doc
|
||||
print-str printf println println-str prn prn-str promise proxy
|
||||
proxy-mappings proxy-super push-thread-bindings pvalues quot rand
|
||||
rand-int range ratio? rationalize re-find re-groups re-matcher
|
||||
re-matches re-pattern re-seq read read-line read-string reduce ref
|
||||
ref-history-count ref-max-history ref-min-history ref-set refer
|
||||
refer-clojure reify release-pending-sends rem remove remove-all-methods
|
||||
remove-method remove-ns remove-watch repeat repeatedly replace replicate
|
||||
require reset! reset-meta! resolve rest restart-agent resultset-seq
|
||||
reverse reversible? rseq rsubseq satisfies? second select-keys send
|
||||
send-off seq seq? seque sequence sequential? set set-error-handler!
|
||||
set-error-mode! set-validator! set? short short-array shorts
|
||||
shutdown-agents slurp some sort sort-by sorted-map sorted-map-by
|
||||
sorted-set sorted-set-by sorted? special-form-anchor special-symbol?
|
||||
split-at split-with str string? struct struct-map subs subseq subvec
|
||||
supers swap! symbol symbol? sync syntax-symbol-anchor take take-last
|
||||
take-nth take-while test the-ns thread-bound? time to-array to-array-2d
|
||||
trampoline transient tree-seq true? type unchecked-add unchecked-dec
|
||||
unchecked-divide unchecked-inc unchecked-multiply unchecked-negate
|
||||
unchecked-remainder unchecked-subtract underive update-in update-proxy
|
||||
use val vals var-get var-set var? vary-meta vec vector vector-of vector?
|
||||
when when-first when-let when-not while with-bindings with-bindings*
|
||||
with-in-str with-local-vars with-meta with-open with-out-str
|
||||
with-precision xml-seq zero? zipmap
|
||||
] # :nodoc:
|
||||
|
||||
PREDEFINED_CONSTANTS = %w[
|
||||
true false nil *1 *2 *3 *agent* *clojure-version* *command-line-args*
|
||||
*compile-files* *compile-path* *e *err* *file* *flush-on-newline*
|
||||
*in* *ns* *out* *print-dup* *print-length* *print-level* *print-meta*
|
||||
*print-readably* *read-eval* *warn-on-reflection*
|
||||
] # :nodoc:
|
||||
|
||||
IDENT_KIND = WordList.new(:ident).
|
||||
add(SPECIAL_FORMS, :keyword).
|
||||
add(CORE_FORMS, :keyword).
|
||||
add(PREDEFINED_CONSTANTS, :predefined_constant)
|
||||
|
||||
KEYWORD_NEXT_TOKEN_KIND = WordList.new(nil).
|
||||
add(%w[ def defn defn- definline defmacro defmulti defmethod defstruct defonce declare ], :function).
|
||||
add(%w[ ns ], :namespace).
|
||||
add(%w[ defprotocol defrecord ], :class)
|
||||
|
||||
BASIC_IDENTIFIER = /[a-zA-Z$%*\/_+!?&<>\-=]=?[a-zA-Z0-9$&*+!\/_?<>\-\#]*/
|
||||
IDENTIFIER = /(?!-\d)(?:(?:#{BASIC_IDENTIFIER}\.)*#{BASIC_IDENTIFIER}(?:\/#{BASIC_IDENTIFIER})?\.?)|\.\.?/
|
||||
SYMBOL = /::?#{IDENTIFIER}/o
|
||||
DIGIT = /\d/
|
||||
DIGIT10 = DIGIT
|
||||
DIGIT16 = /[0-9a-f]/i
|
||||
DIGIT8 = /[0-7]/
|
||||
DIGIT2 = /[01]/
|
||||
RADIX16 = /\#x/i
|
||||
RADIX8 = /\#o/i
|
||||
RADIX2 = /\#b/i
|
||||
RADIX10 = /\#d/i
|
||||
EXACTNESS = /#i|#e/i
|
||||
SIGN = /[\+-]?/
|
||||
EXP_MARK = /[esfdl]/i
|
||||
EXP = /#{EXP_MARK}#{SIGN}#{DIGIT}+/
|
||||
SUFFIX = /#{EXP}?/
|
||||
PREFIX10 = /#{RADIX10}?#{EXACTNESS}?|#{EXACTNESS}?#{RADIX10}?/
|
||||
PREFIX16 = /#{RADIX16}#{EXACTNESS}?|#{EXACTNESS}?#{RADIX16}/
|
||||
PREFIX8 = /#{RADIX8}#{EXACTNESS}?|#{EXACTNESS}?#{RADIX8}/
|
||||
PREFIX2 = /#{RADIX2}#{EXACTNESS}?|#{EXACTNESS}?#{RADIX2}/
|
||||
UINT10 = /#{DIGIT10}+#*/
|
||||
UINT16 = /#{DIGIT16}+#*/
|
||||
UINT8 = /#{DIGIT8}+#*/
|
||||
UINT2 = /#{DIGIT2}+#*/
|
||||
DECIMAL = /#{DIGIT10}+#+\.#*#{SUFFIX}|#{DIGIT10}+\.#{DIGIT10}*#*#{SUFFIX}|\.#{DIGIT10}+#*#{SUFFIX}|#{UINT10}#{EXP}/
|
||||
UREAL10 = /#{UINT10}\/#{UINT10}|#{DECIMAL}|#{UINT10}/
|
||||
UREAL16 = /#{UINT16}\/#{UINT16}|#{UINT16}/
|
||||
UREAL8 = /#{UINT8}\/#{UINT8}|#{UINT8}/
|
||||
UREAL2 = /#{UINT2}\/#{UINT2}|#{UINT2}/
|
||||
REAL10 = /#{SIGN}#{UREAL10}/
|
||||
REAL16 = /#{SIGN}#{UREAL16}/
|
||||
REAL8 = /#{SIGN}#{UREAL8}/
|
||||
REAL2 = /#{SIGN}#{UREAL2}/
|
||||
IMAG10 = /i|#{UREAL10}i/
|
||||
IMAG16 = /i|#{UREAL16}i/
|
||||
IMAG8 = /i|#{UREAL8}i/
|
||||
IMAG2 = /i|#{UREAL2}i/
|
||||
COMPLEX10 = /#{REAL10}@#{REAL10}|#{REAL10}\+#{IMAG10}|#{REAL10}-#{IMAG10}|\+#{IMAG10}|-#{IMAG10}|#{REAL10}/
|
||||
COMPLEX16 = /#{REAL16}@#{REAL16}|#{REAL16}\+#{IMAG16}|#{REAL16}-#{IMAG16}|\+#{IMAG16}|-#{IMAG16}|#{REAL16}/
|
||||
COMPLEX8 = /#{REAL8}@#{REAL8}|#{REAL8}\+#{IMAG8}|#{REAL8}-#{IMAG8}|\+#{IMAG8}|-#{IMAG8}|#{REAL8}/
|
||||
COMPLEX2 = /#{REAL2}@#{REAL2}|#{REAL2}\+#{IMAG2}|#{REAL2}-#{IMAG2}|\+#{IMAG2}|-#{IMAG2}|#{REAL2}/
|
||||
NUM10 = /#{PREFIX10}?#{COMPLEX10}/
|
||||
NUM16 = /#{PREFIX16}#{COMPLEX16}/
|
||||
NUM8 = /#{PREFIX8}#{COMPLEX8}/
|
||||
NUM2 = /#{PREFIX2}#{COMPLEX2}/
|
||||
NUM = /#{NUM10}|#{NUM16}|#{NUM8}|#{NUM2}/
|
||||
|
||||
protected
|
||||
|
||||
def scan_tokens encoder, options
|
||||
|
||||
state = :initial
|
||||
kind = nil
|
||||
|
||||
until eos?
|
||||
|
||||
case state
|
||||
when :initial
|
||||
if match = scan(/ \s+ | \\\n | , /x)
|
||||
encoder.text_token match, :space
|
||||
elsif match = scan(/['`\(\[\)\]\{\}]|\#[({]|~@?|[@\^]/)
|
||||
encoder.text_token match, :operator
|
||||
elsif match = scan(/;.*/)
|
||||
encoder.text_token match, :comment # TODO: recognize (comment ...) too
|
||||
elsif match = scan(/\#?\\(?:newline|space|.?)/)
|
||||
encoder.text_token match, :char
|
||||
elsif match = scan(/\#[ft]/)
|
||||
encoder.text_token match, :predefined_constant
|
||||
elsif match = scan(/#{IDENTIFIER}/o)
|
||||
kind = IDENT_KIND[match]
|
||||
encoder.text_token match, kind
|
||||
if rest? && kind == :keyword
|
||||
if kind = KEYWORD_NEXT_TOKEN_KIND[match]
|
||||
encoder.text_token match, :space if match = scan(/\s+/o)
|
||||
encoder.text_token match, kind if match = scan(/#{IDENTIFIER}/o)
|
||||
end
|
||||
end
|
||||
elsif match = scan(/#{SYMBOL}/o)
|
||||
encoder.text_token match, :symbol
|
||||
elsif match = scan(/\./)
|
||||
encoder.text_token match, :operator
|
||||
elsif match = scan(/ \# \^ #{IDENTIFIER} /ox)
|
||||
encoder.text_token match, :type
|
||||
elsif match = scan(/ (\#)? " /x)
|
||||
state = self[1] ? :regexp : :string
|
||||
encoder.begin_group state
|
||||
encoder.text_token match, :delimiter
|
||||
elsif match = scan(/#{NUM}/o) and not matched.empty?
|
||||
encoder.text_token match, match[/[.e\/]/i] ? :float : :integer
|
||||
else
|
||||
encoder.text_token getch, :error
|
||||
end
|
||||
|
||||
when :string, :regexp
|
||||
if match = scan(/[^"\\]+|\\.?/)
|
||||
encoder.text_token match, :content
|
||||
elsif match = scan(/"/)
|
||||
encoder.text_token match, :delimiter
|
||||
encoder.end_group state
|
||||
state = :initial
|
||||
else
|
||||
raise_inspect "else case \" reached; %p not handled." % peek(1),
|
||||
encoder, state
|
||||
end
|
||||
|
||||
else
|
||||
raise 'else case reached'
|
||||
|
||||
end
|
||||
|
||||
end
|
||||
|
||||
if [:string, :regexp].include? state
|
||||
encoder.end_group state
|
||||
end
|
||||
|
||||
encoder
|
||||
|
||||
end
|
||||
end
|
||||
end
|
||||
end
|
||||
|
|
@ -0,0 +1,215 @@
|
|||
module CodeRay
|
||||
module Scanners
|
||||
|
||||
# Scanner for C++.
|
||||
#
|
||||
# Aliases: +cplusplus+, c++
|
||||
class CPlusPlus < Scanner
|
||||
|
||||
register_for :cpp
|
||||
file_extension 'cpp'
|
||||
title 'C++'
|
||||
|
||||
#-- http://www.cppreference.com/wiki/keywords/start
|
||||
KEYWORDS = [
|
||||
'and', 'and_eq', 'asm', 'bitand', 'bitor', 'break',
|
||||
'case', 'catch', 'class', 'compl', 'const_cast',
|
||||
'continue', 'default', 'delete', 'do', 'dynamic_cast', 'else',
|
||||
'enum', 'export', 'for', 'goto', 'if', 'namespace', 'new',
|
||||
'not', 'not_eq', 'or', 'or_eq', 'reinterpret_cast', 'return',
|
||||
'sizeof', 'static_cast', 'struct', 'switch', 'template',
|
||||
'throw', 'try', 'typedef', 'typeid', 'typename', 'union',
|
||||
'while', 'xor', 'xor_eq',
|
||||
] # :nodoc:
|
||||
|
||||
PREDEFINED_TYPES = [
|
||||
'bool', 'char', 'double', 'float', 'int', 'long',
|
||||
'short', 'signed', 'unsigned', 'wchar_t', 'string',
|
||||
] # :nodoc:
|
||||
PREDEFINED_CONSTANTS = [
|
||||
'false', 'true',
|
||||
'EOF', 'NULL',
|
||||
] # :nodoc:
|
||||
PREDEFINED_VARIABLES = [
|
||||
'this',
|
||||
] # :nodoc:
|
||||
DIRECTIVES = [
|
||||
'auto', 'const', 'explicit', 'extern', 'friend', 'inline', 'mutable', 'operator',
|
||||
'private', 'protected', 'public', 'register', 'static', 'using', 'virtual', 'void',
|
||||
'volatile',
|
||||
] # :nodoc:
|
||||
|
||||
IDENT_KIND = WordList.new(:ident).
|
||||
add(KEYWORDS, :keyword).
|
||||
add(PREDEFINED_TYPES, :predefined_type).
|
||||
add(PREDEFINED_VARIABLES, :local_variable).
|
||||
add(DIRECTIVES, :directive).
|
||||
add(PREDEFINED_CONSTANTS, :predefined_constant) # :nodoc:
|
||||
|
||||
ESCAPE = / [rbfntv\n\\'"] | x[a-fA-F0-9]{1,2} | [0-7]{1,3} /x # :nodoc:
|
||||
UNICODE_ESCAPE = / u[a-fA-F0-9]{4} | U[a-fA-F0-9]{8} /x # :nodoc:
|
||||
|
||||
protected
|
||||
|
||||
def scan_tokens encoder, options
|
||||
|
||||
state = :initial
|
||||
label_expected = true
|
||||
case_expected = false
|
||||
label_expected_before_preproc_line = nil
|
||||
in_preproc_line = false
|
||||
|
||||
until eos?
|
||||
|
||||
case state
|
||||
|
||||
when :initial
|
||||
|
||||
if match = scan(/ \s+ | \\\n /x)
|
||||
if in_preproc_line && match != "\\\n" && match.index(?\n)
|
||||
in_preproc_line = false
|
||||
label_expected = label_expected_before_preproc_line
|
||||
end
|
||||
encoder.text_token match, :space
|
||||
|
||||
elsif match = scan(%r! // [^\n\\]* (?: \\. [^\n\\]* )* | /\* (?: .*? \*/ | .* ) !mx)
|
||||
encoder.text_token match, :comment
|
||||
|
||||
elsif match = scan(/ \# \s* if \s* 0 /x)
|
||||
match << scan_until(/ ^\# (?:elif|else|endif) .*? $ | \z /xm) unless eos?
|
||||
encoder.text_token match, :comment
|
||||
|
||||
elsif match = scan(/ [-+*=<>?:;,!&^|()\[\]{}~%]+ | \/=? | \.(?!\d) /x)
|
||||
label_expected = match =~ /[;\{\}]/
|
||||
if case_expected
|
||||
label_expected = true if match == ':'
|
||||
case_expected = false
|
||||
end
|
||||
encoder.text_token match, :operator
|
||||
|
||||
elsif match = scan(/ [A-Za-z_][A-Za-z_0-9]* /x)
|
||||
kind = IDENT_KIND[match]
|
||||
if kind == :ident && label_expected && !in_preproc_line && scan(/:(?!:)/)
|
||||
kind = :label
|
||||
match << matched
|
||||
else
|
||||
label_expected = false
|
||||
if kind == :keyword
|
||||
case match
|
||||
when 'class'
|
||||
state = :class_name_expected
|
||||
when 'case', 'default'
|
||||
case_expected = true
|
||||
end
|
||||
end
|
||||
end
|
||||
encoder.text_token match, kind
|
||||
|
||||
elsif match = scan(/\$/)
|
||||
encoder.text_token match, :ident
|
||||
|
||||
elsif match = scan(/L?"/)
|
||||
encoder.begin_group :string
|
||||
if match[0] == ?L
|
||||
encoder.text_token match, 'L', :modifier
|
||||
match = '"'
|
||||
end
|
||||
state = :string
|
||||
encoder.text_token match, :delimiter
|
||||
|
||||
elsif match = scan(/#[ \t]*(\w*)/)
|
||||
encoder.text_token match, :preprocessor
|
||||
in_preproc_line = true
|
||||
label_expected_before_preproc_line = label_expected
|
||||
state = :include_expected if self[1] == 'include'
|
||||
|
||||
elsif match = scan(/ L?' (?: [^\'\n\\] | \\ #{ESCAPE} )? '? /ox)
|
||||
label_expected = false
|
||||
encoder.text_token match, :char
|
||||
|
||||
elsif match = scan(/0[xX][0-9A-Fa-f]+/)
|
||||
label_expected = false
|
||||
encoder.text_token match, :hex
|
||||
|
||||
elsif match = scan(/(?:0[0-7]+)(?![89.eEfF])/)
|
||||
label_expected = false
|
||||
encoder.text_token match, :octal
|
||||
|
||||
elsif match = scan(/(?:\d+)(?![.eEfF])L?L?/)
|
||||
label_expected = false
|
||||
encoder.text_token match, :integer
|
||||
|
||||
elsif match = scan(/\d[fF]?|\d*\.\d+(?:[eE][+-]?\d+)?[fF]?|\d+[eE][+-]?\d+[fF]?/)
|
||||
label_expected = false
|
||||
encoder.text_token match, :float
|
||||
|
||||
else
|
||||
encoder.text_token getch, :error
|
||||
|
||||
end
|
||||
|
||||
when :string
|
||||
if match = scan(/[^\\"]+/)
|
||||
encoder.text_token match, :content
|
||||
elsif match = scan(/"/)
|
||||
encoder.text_token match, :delimiter
|
||||
encoder.end_group :string
|
||||
state = :initial
|
||||
label_expected = false
|
||||
elsif match = scan(/ \\ (?: #{ESCAPE} | #{UNICODE_ESCAPE} ) /mox)
|
||||
encoder.text_token match, :char
|
||||
elsif match = scan(/ \\ | $ /x)
|
||||
encoder.end_group :string
|
||||
encoder.text_token match, :error
|
||||
state = :initial
|
||||
label_expected = false
|
||||
else
|
||||
raise_inspect "else case \" reached; %p not handled." % peek(1), encoder
|
||||
end
|
||||
|
||||
when :include_expected
|
||||
if match = scan(/<[^>\n]+>?|"[^"\n\\]*(?:\\.[^"\n\\]*)*"?/)
|
||||
encoder.text_token match, :include
|
||||
state = :initial
|
||||
|
||||
elsif match = scan(/\s+/)
|
||||
encoder.text_token match, :space
|
||||
state = :initial if match.index ?\n
|
||||
|
||||
else
|
||||
state = :initial
|
||||
|
||||
end
|
||||
|
||||
when :class_name_expected
|
||||
if match = scan(/ [A-Za-z_][A-Za-z_0-9]* /x)
|
||||
encoder.text_token match, :class
|
||||
state = :initial
|
||||
|
||||
elsif match = scan(/\s+/)
|
||||
encoder.text_token match, :space
|
||||
|
||||
else
|
||||
encoder.text_token getch, :error
|
||||
state = :initial
|
||||
|
||||
end
|
||||
|
||||
else
|
||||
raise_inspect 'Unknown state', encoder
|
||||
|
||||
end
|
||||
|
||||
end
|
||||
|
||||
if state == :string
|
||||
encoder.end_group :string
|
||||
end
|
||||
|
||||
encoder
|
||||
end
|
||||
|
||||
end
|
||||
|
||||
end
|
||||
end
|
||||
|
|
@ -0,0 +1,199 @@
|
|||
module CodeRay
|
||||
module Scanners
|
||||
|
||||
class CSS < Scanner
|
||||
|
||||
register_for :css
|
||||
|
||||
KINDS_NOT_LOC = [
|
||||
:comment,
|
||||
:class, :pseudo_class, :type,
|
||||
:constant, :directive,
|
||||
:key, :value, :operator, :color, :float, :string,
|
||||
:error, :important,
|
||||
] # :nodoc:
|
||||
|
||||
module RE # :nodoc:
|
||||
Hex = /[0-9a-fA-F]/
|
||||
Unicode = /\\#{Hex}{1,6}(?:\r\n|\s)?/ # differs from standard because it allows uppercase hex too
|
||||
Escape = /#{Unicode}|\\[^\r\n\f0-9a-fA-F]/
|
||||
NMChar = /[-_a-zA-Z0-9]|#{Escape}/
|
||||
NMStart = /[_a-zA-Z]|#{Escape}/
|
||||
NL = /\r\n|\r|\n|\f/
|
||||
String1 = /"(?:[^\n\r\f\\"]|\\#{NL}|#{Escape})*"?/ # TODO: buggy regexp
|
||||
String2 = /'(?:[^\n\r\f\\']|\\#{NL}|#{Escape})*'?/ # TODO: buggy regexp
|
||||
String = /#{String1}|#{String2}/
|
||||
|
||||
HexColor = /#(?:#{Hex}{6}|#{Hex}{3})/
|
||||
Color = /#{HexColor}/
|
||||
|
||||
Num = /-?(?:[0-9]+|[0-9]*\.[0-9]+)/
|
||||
Name = /#{NMChar}+/
|
||||
Ident = /-?#{NMStart}#{NMChar}*/
|
||||
AtKeyword = /@#{Ident}/
|
||||
Percentage = /#{Num}%/
|
||||
|
||||
reldimensions = %w[em ex px]
|
||||
absdimensions = %w[in cm mm pt pc]
|
||||
Unit = Regexp.union(*(reldimensions + absdimensions + %w[s]))
|
||||
|
||||
Dimension = /#{Num}#{Unit}/
|
||||
|
||||
Comment = %r! /\* (?: .*? \*/ | .* ) !mx
|
||||
Function = /(?:url|alpha|attr|counters?)\((?:[^)\n\r\f]|\\\))*\)?/
|
||||
|
||||
Id = /##{Name}/
|
||||
Class = /\.#{Name}/
|
||||
PseudoClass = /:#{Name}/
|
||||
AttributeSelector = /\[[^\]]*\]?/
|
||||
end
|
||||
|
||||
protected
|
||||
|
||||
def setup
|
||||
@state = :initial
|
||||
@value_expected = nil
|
||||
end
|
||||
|
||||
def scan_tokens encoder, options
|
||||
states = Array(options[:state] || @state)
|
||||
value_expected = @value_expected
|
||||
|
||||
until eos?
|
||||
|
||||
if match = scan(/\s+/)
|
||||
encoder.text_token match, :space
|
||||
|
||||
elsif case states.last
|
||||
when :initial, :media
|
||||
if match = scan(/(?>#{RE::Ident})(?!\()|\*/ox)
|
||||
encoder.text_token match, :type
|
||||
next
|
||||
elsif match = scan(RE::Class)
|
||||
encoder.text_token match, :class
|
||||
next
|
||||
elsif match = scan(RE::Id)
|
||||
encoder.text_token match, :constant
|
||||
next
|
||||
elsif match = scan(RE::PseudoClass)
|
||||
encoder.text_token match, :pseudo_class
|
||||
next
|
||||
elsif match = scan(RE::AttributeSelector)
|
||||
# TODO: Improve highlighting inside of attribute selectors.
|
||||
encoder.text_token match[0,1], :operator
|
||||
encoder.text_token match[1..-2], :attribute_name if match.size > 2
|
||||
encoder.text_token match[-1,1], :operator if match[-1] == ?]
|
||||
next
|
||||
elsif match = scan(/@media/)
|
||||
encoder.text_token match, :directive
|
||||
states.push :media_before_name
|
||||
next
|
||||
end
|
||||
|
||||
when :block
|
||||
if match = scan(/(?>#{RE::Ident})(?!\()/ox)
|
||||
if value_expected
|
||||
encoder.text_token match, :value
|
||||
else
|
||||
encoder.text_token match, :key
|
||||
end
|
||||
next
|
||||
end
|
||||
|
||||
when :media_before_name
|
||||
if match = scan(RE::Ident)
|
||||
encoder.text_token match, :type
|
||||
states[-1] = :media_after_name
|
||||
next
|
||||
end
|
||||
|
||||
when :media_after_name
|
||||
if match = scan(/\{/)
|
||||
encoder.text_token match, :operator
|
||||
states[-1] = :media
|
||||
next
|
||||
end
|
||||
|
||||
else
|
||||
#:nocov:
|
||||
raise_inspect 'Unknown state', encoder
|
||||
#:nocov:
|
||||
|
||||
end
|
||||
|
||||
elsif match = scan(/\/\*(?:.*?\*\/|\z)/m)
|
||||
encoder.text_token match, :comment
|
||||
|
||||
elsif match = scan(/\{/)
|
||||
value_expected = false
|
||||
encoder.text_token match, :operator
|
||||
states.push :block
|
||||
|
||||
elsif match = scan(/\}/)
|
||||
value_expected = false
|
||||
encoder.text_token match, :operator
|
||||
if states.last == :block || states.last == :media
|
||||
states.pop
|
||||
end
|
||||
|
||||
elsif match = scan(/#{RE::String}/o)
|
||||
encoder.begin_group :string
|
||||
encoder.text_token match[0, 1], :delimiter
|
||||
encoder.text_token match[1..-2], :content if match.size > 2
|
||||
encoder.text_token match[-1, 1], :delimiter if match.size >= 2
|
||||
encoder.end_group :string
|
||||
|
||||
elsif match = scan(/#{RE::Function}/o)
|
||||
encoder.begin_group :function
|
||||
start = match[/^\w+\(/]
|
||||
encoder.text_token start, :delimiter
|
||||
if match[-1] == ?)
|
||||
encoder.text_token match[start.size..-2], :content
|
||||
encoder.text_token ')', :delimiter
|
||||
else
|
||||
encoder.text_token match[start.size..-1], :content
|
||||
end
|
||||
encoder.end_group :function
|
||||
|
||||
elsif match = scan(/(?: #{RE::Dimension} | #{RE::Percentage} | #{RE::Num} )/ox)
|
||||
encoder.text_token match, :float
|
||||
|
||||
elsif match = scan(/#{RE::Color}/o)
|
||||
encoder.text_token match, :color
|
||||
|
||||
elsif match = scan(/! *important/)
|
||||
encoder.text_token match, :important
|
||||
|
||||
elsif match = scan(/(?:rgb|hsl)a?\([^()\n]*\)?/)
|
||||
encoder.text_token match, :color
|
||||
|
||||
elsif match = scan(RE::AtKeyword)
|
||||
encoder.text_token match, :directive
|
||||
|
||||
elsif match = scan(/ [+>:;,.=()\/] /x)
|
||||
if match == ':'
|
||||
value_expected = true
|
||||
elsif match == ';'
|
||||
value_expected = false
|
||||
end
|
||||
encoder.text_token match, :operator
|
||||
|
||||
else
|
||||
encoder.text_token getch, :error
|
||||
|
||||
end
|
||||
|
||||
end
|
||||
|
||||
if options[:keep_state]
|
||||
@state = states
|
||||
@value_expected = value_expected
|
||||
end
|
||||
|
||||
encoder
|
||||
end
|
||||
|
||||
end
|
||||
|
||||
end
|
||||
end
|
||||
|
|
@ -0,0 +1,65 @@
|
|||
module CodeRay
|
||||
module Scanners
|
||||
|
||||
# = Debug Scanner
|
||||
#
|
||||
# Interprets the output of the Encoders::Debug encoder.
|
||||
class Debug < Scanner
|
||||
|
||||
register_for :debug
|
||||
title 'CodeRay Token Dump Import'
|
||||
|
||||
protected
|
||||
|
||||
def scan_tokens encoder, options
|
||||
|
||||
opened_tokens = []
|
||||
|
||||
until eos?
|
||||
|
||||
if match = scan(/\s+/)
|
||||
encoder.text_token match, :space
|
||||
|
||||
elsif match = scan(/ (\w+) \( ( [^\)\\]* ( \\. [^\)\\]* )* ) \)? /x)
|
||||
kind = self[1].to_sym
|
||||
match = self[2].gsub(/\\(.)/m, '\1')
|
||||
unless TokenKinds.has_key? kind
|
||||
kind = :error
|
||||
match = matched
|
||||
end
|
||||
encoder.text_token match, kind
|
||||
|
||||
elsif match = scan(/ (\w+) ([<\[]) /x)
|
||||
kind = self[1].to_sym
|
||||
opened_tokens << kind
|
||||
case self[2]
|
||||
when '<'
|
||||
encoder.begin_group kind
|
||||
when '['
|
||||
encoder.begin_line kind
|
||||
else
|
||||
raise 'CodeRay bug: This case should not be reached.'
|
||||
end
|
||||
|
||||
elsif !opened_tokens.empty? && match = scan(/ > /x)
|
||||
encoder.end_group opened_tokens.pop
|
||||
|
||||
elsif !opened_tokens.empty? && match = scan(/ \] /x)
|
||||
encoder.end_line opened_tokens.pop
|
||||
|
||||
else
|
||||
encoder.text_token getch, :space
|
||||
|
||||
end
|
||||
|
||||
end
|
||||
|
||||
encoder.end_group opened_tokens.pop until opened_tokens.empty?
|
||||
|
||||
encoder
|
||||
end
|
||||
|
||||
end
|
||||
|
||||
end
|
||||
end
|
||||
|
|
@ -0,0 +1,144 @@
|
|||
module CodeRay
|
||||
module Scanners
|
||||
|
||||
# Scanner for the Delphi language (Object Pascal).
|
||||
#
|
||||
# Alias: +pascal+
|
||||
class Delphi < Scanner
|
||||
|
||||
register_for :delphi
|
||||
file_extension 'pas'
|
||||
|
||||
KEYWORDS = [
|
||||
'and', 'array', 'as', 'at', 'asm', 'at', 'begin', 'case', 'class',
|
||||
'const', 'constructor', 'destructor', 'dispinterface', 'div', 'do',
|
||||
'downto', 'else', 'end', 'except', 'exports', 'file', 'finalization',
|
||||
'finally', 'for', 'function', 'goto', 'if', 'implementation', 'in',
|
||||
'inherited', 'initialization', 'inline', 'interface', 'is', 'label',
|
||||
'library', 'mod', 'nil', 'not', 'object', 'of', 'or', 'out', 'packed',
|
||||
'procedure', 'program', 'property', 'raise', 'record', 'repeat',
|
||||
'resourcestring', 'set', 'shl', 'shr', 'string', 'then', 'threadvar',
|
||||
'to', 'try', 'type', 'unit', 'until', 'uses', 'var', 'while', 'with',
|
||||
'xor', 'on',
|
||||
] # :nodoc:
|
||||
|
||||
DIRECTIVES = [
|
||||
'absolute', 'abstract', 'assembler', 'at', 'automated', 'cdecl',
|
||||
'contains', 'deprecated', 'dispid', 'dynamic', 'export',
|
||||
'external', 'far', 'forward', 'implements', 'local',
|
||||
'near', 'nodefault', 'on', 'overload', 'override',
|
||||
'package', 'pascal', 'platform', 'private', 'protected', 'public',
|
||||
'published', 'read', 'readonly', 'register', 'reintroduce',
|
||||
'requires', 'resident', 'safecall', 'stdcall', 'stored', 'varargs',
|
||||
'virtual', 'write', 'writeonly',
|
||||
] # :nodoc:
|
||||
|
||||
IDENT_KIND = WordList::CaseIgnoring.new(:ident).
|
||||
add(KEYWORDS, :keyword).
|
||||
add(DIRECTIVES, :directive) # :nodoc:
|
||||
|
||||
NAME_FOLLOWS = WordList::CaseIgnoring.new(false).
|
||||
add(%w(procedure function .)) # :nodoc:
|
||||
|
||||
protected
|
||||
|
||||
def scan_tokens encoder, options
|
||||
|
||||
state = :initial
|
||||
last_token = ''
|
||||
|
||||
until eos?
|
||||
|
||||
if state == :initial
|
||||
|
||||
if match = scan(/ \s+ /x)
|
||||
encoder.text_token match, :space
|
||||
next
|
||||
|
||||
elsif match = scan(%r! \{ \$ [^}]* \}? | \(\* \$ (?: .*? \*\) | .* ) !mx)
|
||||
encoder.text_token match, :preprocessor
|
||||
next
|
||||
|
||||
elsif match = scan(%r! // [^\n]* | \{ [^}]* \}? | \(\* (?: .*? \*\) | .* ) !mx)
|
||||
encoder.text_token match, :comment
|
||||
next
|
||||
|
||||
elsif match = scan(/ <[>=]? | >=? | :=? | [-+=*\/;,@\^|\(\)\[\]] | \.\. /x)
|
||||
encoder.text_token match, :operator
|
||||
|
||||
elsif match = scan(/\./)
|
||||
encoder.text_token match, :operator
|
||||
next if last_token == 'end'
|
||||
|
||||
elsif match = scan(/ [A-Za-z_][A-Za-z_0-9]* /x)
|
||||
encoder.text_token match, NAME_FOLLOWS[last_token] ? :ident : IDENT_KIND[match]
|
||||
|
||||
elsif match = skip(/ ' ( [^\n']|'' ) (?:'|$) /x)
|
||||
encoder.begin_group :char
|
||||
encoder.text_token "'", :delimiter
|
||||
encoder.text_token self[1], :content
|
||||
encoder.text_token "'", :delimiter
|
||||
encoder.end_group :char
|
||||
next
|
||||
|
||||
elsif match = scan(/ ' /x)
|
||||
encoder.begin_group :string
|
||||
encoder.text_token match, :delimiter
|
||||
state = :string
|
||||
|
||||
elsif match = scan(/ \# (?: \d+ | \$[0-9A-Fa-f]+ ) /x)
|
||||
encoder.text_token match, :char
|
||||
|
||||
elsif match = scan(/ \$ [0-9A-Fa-f]+ /x)
|
||||
encoder.text_token match, :hex
|
||||
|
||||
elsif match = scan(/ (?: \d+ ) (?![eE]|\.[^.]) /x)
|
||||
encoder.text_token match, :integer
|
||||
|
||||
elsif match = scan(/ \d+ (?: \.\d+ (?: [eE][+-]? \d+ )? | [eE][+-]? \d+ ) /x)
|
||||
encoder.text_token match, :float
|
||||
|
||||
else
|
||||
encoder.text_token getch, :error
|
||||
next
|
||||
|
||||
end
|
||||
|
||||
elsif state == :string
|
||||
if match = scan(/[^\n']+/)
|
||||
encoder.text_token match, :content
|
||||
elsif match = scan(/''/)
|
||||
encoder.text_token match, :char
|
||||
elsif match = scan(/'/)
|
||||
encoder.text_token match, :delimiter
|
||||
encoder.end_group :string
|
||||
state = :initial
|
||||
next
|
||||
elsif match = scan(/\n/)
|
||||
encoder.end_group :string
|
||||
encoder.text_token match, :space
|
||||
state = :initial
|
||||
else
|
||||
raise "else case \' reached; %p not handled." % peek(1), encoder
|
||||
end
|
||||
|
||||
else
|
||||
raise 'else-case reached', encoder
|
||||
|
||||
end
|
||||
|
||||
last_token = match
|
||||
|
||||
end
|
||||
|
||||
if state == :string
|
||||
encoder.end_group state
|
||||
end
|
||||
|
||||
encoder
|
||||
end
|
||||
|
||||
end
|
||||
|
||||
end
|
||||
end
|
||||
|
|
@ -0,0 +1,199 @@
|
|||
module CodeRay
|
||||
module Scanners
|
||||
|
||||
# Scanner for output of the diff command.
|
||||
#
|
||||
# Alias: +patch+
|
||||
class Diff < Scanner
|
||||
|
||||
register_for :diff
|
||||
title 'diff output'
|
||||
|
||||
DEFAULT_OPTIONS = {
|
||||
:highlight_code => true,
|
||||
:inline_diff => true,
|
||||
}
|
||||
|
||||
protected
|
||||
|
||||
def scan_tokens encoder, options
|
||||
|
||||
line_kind = nil
|
||||
state = :initial
|
||||
deleted_lines = 0
|
||||
scanners = Hash.new do |h, lang|
|
||||
h[lang] = Scanners[lang].new '', :keep_tokens => true, :keep_state => true
|
||||
end
|
||||
content_scanner = scanners[:plain]
|
||||
content_scanner_entry_state = nil
|
||||
|
||||
until eos?
|
||||
|
||||
if match = scan(/\n/)
|
||||
deleted_lines = 0 unless line_kind == :delete
|
||||
if line_kind
|
||||
encoder.end_line line_kind
|
||||
line_kind = nil
|
||||
end
|
||||
encoder.text_token match, :space
|
||||
next
|
||||
end
|
||||
|
||||
case state
|
||||
|
||||
when :initial
|
||||
if match = scan(/--- |\+\+\+ |=+|_+/)
|
||||
encoder.begin_line line_kind = :head
|
||||
encoder.text_token match, :head
|
||||
if match = scan(/.*?(?=$|[\t\n\x00]| \(revision)/)
|
||||
encoder.text_token match, :filename
|
||||
if options[:highlight_code] && match != '/dev/null'
|
||||
file_type = CodeRay::FileType.fetch(match, :text)
|
||||
file_type = :text if file_type == :diff
|
||||
content_scanner = scanners[file_type]
|
||||
content_scanner_entry_state = nil
|
||||
end
|
||||
end
|
||||
next unless match = scan(/.+/)
|
||||
encoder.text_token match, :plain
|
||||
elsif match = scan(/Index: |Property changes on: /)
|
||||
encoder.begin_line line_kind = :head
|
||||
encoder.text_token match, :head
|
||||
next unless match = scan(/.+/)
|
||||
encoder.text_token match, :plain
|
||||
elsif match = scan(/Added: /)
|
||||
encoder.begin_line line_kind = :head
|
||||
encoder.text_token match, :head
|
||||
next unless match = scan(/.+/)
|
||||
encoder.text_token match, :plain
|
||||
state = :added
|
||||
elsif match = scan(/\\ .*/)
|
||||
encoder.text_token match, :comment
|
||||
elsif match = scan(/@@(?>[^@\n]*)@@/)
|
||||
content_scanner.state = :initial unless match?(/\n\+/)
|
||||
content_scanner_entry_state = nil
|
||||
if check(/\n|$/)
|
||||
encoder.begin_line line_kind = :change
|
||||
else
|
||||
encoder.begin_group :change
|
||||
end
|
||||
encoder.text_token match[0,2], :change
|
||||
encoder.text_token match[2...-2], :plain
|
||||
encoder.text_token match[-2,2], :change
|
||||
encoder.end_group :change unless line_kind
|
||||
next unless match = scan(/.+/)
|
||||
if options[:highlight_code]
|
||||
content_scanner.tokenize match, :tokens => encoder
|
||||
else
|
||||
encoder.text_token match, :plain
|
||||
end
|
||||
next
|
||||
elsif match = scan(/\+/)
|
||||
encoder.begin_line line_kind = :insert
|
||||
encoder.text_token match, :insert
|
||||
next unless match = scan(/.+/)
|
||||
if options[:highlight_code]
|
||||
content_scanner.tokenize match, :tokens => encoder
|
||||
else
|
||||
encoder.text_token match, :plain
|
||||
end
|
||||
next
|
||||
elsif match = scan(/-/)
|
||||
deleted_lines += 1
|
||||
encoder.begin_line line_kind = :delete
|
||||
encoder.text_token match, :delete
|
||||
if options[:inline_diff] && deleted_lines == 1 && check(/(?>.*)\n\+(?>.*)$(?!\n\+)/)
|
||||
content_scanner_entry_state = content_scanner.state
|
||||
skip(/(.*)\n\+(.*)$/)
|
||||
head, deletion, insertion, tail = diff self[1], self[2]
|
||||
pre, deleted, post = content_scanner.tokenize [head, deletion, tail], :tokens => Tokens.new
|
||||
encoder.tokens pre
|
||||
unless deleted.empty?
|
||||
encoder.begin_group :eyecatcher
|
||||
encoder.tokens deleted
|
||||
encoder.end_group :eyecatcher
|
||||
end
|
||||
encoder.tokens post
|
||||
encoder.end_line line_kind
|
||||
encoder.text_token "\n", :space
|
||||
encoder.begin_line line_kind = :insert
|
||||
encoder.text_token '+', :insert
|
||||
content_scanner.state = content_scanner_entry_state || :initial
|
||||
pre, inserted, post = content_scanner.tokenize [head, insertion, tail], :tokens => Tokens.new
|
||||
encoder.tokens pre
|
||||
unless inserted.empty?
|
||||
encoder.begin_group :eyecatcher
|
||||
encoder.tokens inserted
|
||||
encoder.end_group :eyecatcher
|
||||
end
|
||||
encoder.tokens post
|
||||
elsif match = scan(/.*/)
|
||||
if options[:highlight_code]
|
||||
if deleted_lines == 1
|
||||
content_scanner_entry_state = content_scanner.state
|
||||
end
|
||||
content_scanner.tokenize match, :tokens => encoder unless match.empty?
|
||||
if !match?(/\n-/)
|
||||
if match?(/\n\+/)
|
||||
content_scanner.state = content_scanner_entry_state || :initial
|
||||
end
|
||||
content_scanner_entry_state = nil
|
||||
end
|
||||
else
|
||||
encoder.text_token match, :plain
|
||||
end
|
||||
end
|
||||
next
|
||||
elsif match = scan(/ .*/)
|
||||
if options[:highlight_code]
|
||||
content_scanner.tokenize match, :tokens => encoder
|
||||
else
|
||||
encoder.text_token match, :plain
|
||||
end
|
||||
next
|
||||
elsif match = scan(/.+/)
|
||||
encoder.begin_line line_kind = :comment
|
||||
encoder.text_token match, :plain
|
||||
else
|
||||
raise_inspect 'else case rached'
|
||||
end
|
||||
|
||||
when :added
|
||||
if match = scan(/ \+/)
|
||||
encoder.begin_line line_kind = :insert
|
||||
encoder.text_token match, :insert
|
||||
next unless match = scan(/.+/)
|
||||
encoder.text_token match, :plain
|
||||
else
|
||||
state = :initial
|
||||
next
|
||||
end
|
||||
end
|
||||
|
||||
end
|
||||
|
||||
encoder.end_line line_kind if line_kind
|
||||
|
||||
encoder
|
||||
end
|
||||
|
||||
private
|
||||
|
||||
def diff a, b
|
||||
# i will be the index of the leftmost difference from the left.
|
||||
i_max = [a.size, b.size].min
|
||||
i = 0
|
||||
i += 1 while i < i_max && a[i] == b[i]
|
||||
# j_min will be the index of the leftmost difference from the right.
|
||||
j_min = i - i_max
|
||||
# j will be the index of the rightmost difference from the right which
|
||||
# does not precede the leftmost one from the left.
|
||||
j = -1
|
||||
j -= 1 while j >= j_min && a[j] == b[j]
|
||||
return a[0...i], a[i..j], b[i..j], (j < -1) ? a[j+1..-1] : ''
|
||||
end
|
||||
|
||||
end
|
||||
|
||||
end
|
||||
end
|
||||
|
|
@ -0,0 +1,81 @@
|
|||
module CodeRay
|
||||
module Scanners
|
||||
|
||||
load :html
|
||||
load :ruby
|
||||
|
||||
# Scanner for HTML ERB templates.
|
||||
class ERB < Scanner
|
||||
|
||||
register_for :erb
|
||||
title 'HTML ERB Template'
|
||||
|
||||
KINDS_NOT_LOC = HTML::KINDS_NOT_LOC
|
||||
|
||||
ERB_RUBY_BLOCK = /
|
||||
(<%(?!%)[-=\#]?)
|
||||
((?>
|
||||
[^\-%]* # normal*
|
||||
(?> # special
|
||||
(?: %(?!>) | -(?!%>) )
|
||||
[^\-%]* # normal*
|
||||
)*
|
||||
))
|
||||
((?: -?%> )?)
|
||||
/x # :nodoc:
|
||||
|
||||
START_OF_ERB = /
|
||||
<%(?!%)
|
||||
/x # :nodoc:
|
||||
|
||||
protected
|
||||
|
||||
def setup
|
||||
@ruby_scanner = CodeRay.scanner :ruby, :tokens => @tokens, :keep_tokens => true
|
||||
@html_scanner = CodeRay.scanner :html, :tokens => @tokens, :keep_tokens => true, :keep_state => true
|
||||
end
|
||||
|
||||
def reset_instance
|
||||
super
|
||||
@html_scanner.reset
|
||||
end
|
||||
|
||||
def scan_tokens encoder, options
|
||||
|
||||
until eos?
|
||||
|
||||
if (match = scan_until(/(?=#{START_OF_ERB})/o) || scan_rest) and not match.empty?
|
||||
@html_scanner.tokenize match, :tokens => encoder
|
||||
|
||||
elsif match = scan(/#{ERB_RUBY_BLOCK}/o)
|
||||
start_tag = self[1]
|
||||
code = self[2]
|
||||
end_tag = self[3]
|
||||
|
||||
encoder.begin_group :inline
|
||||
encoder.text_token start_tag, :inline_delimiter
|
||||
|
||||
if start_tag == '<%#'
|
||||
encoder.text_token code, :comment
|
||||
else
|
||||
@ruby_scanner.tokenize code, :tokens => encoder
|
||||
end unless code.empty?
|
||||
|
||||
encoder.text_token end_tag, :inline_delimiter unless end_tag.empty?
|
||||
encoder.end_group :inline
|
||||
|
||||
else
|
||||
raise_inspect 'else-case reached!', encoder
|
||||
|
||||
end
|
||||
|
||||
end
|
||||
|
||||
encoder
|
||||
|
||||
end
|
||||
|
||||
end
|
||||
|
||||
end
|
||||
end
|
||||
|
|
@ -0,0 +1,255 @@
|
|||
module CodeRay
|
||||
module Scanners
|
||||
|
||||
load :java
|
||||
|
||||
# Scanner for Groovy.
|
||||
class Groovy < Java
|
||||
|
||||
register_for :groovy
|
||||
|
||||
# TODO: check list of keywords
|
||||
GROOVY_KEYWORDS = %w[
|
||||
as assert def in
|
||||
] # :nodoc:
|
||||
KEYWORDS_EXPECTING_VALUE = WordList.new.add %w[
|
||||
case instanceof new return throw typeof while as assert in
|
||||
] # :nodoc:
|
||||
GROOVY_MAGIC_VARIABLES = %w[ it ] # :nodoc:
|
||||
|
||||
IDENT_KIND = Java::IDENT_KIND.dup.
|
||||
add(GROOVY_KEYWORDS, :keyword).
|
||||
add(GROOVY_MAGIC_VARIABLES, :local_variable) # :nodoc:
|
||||
|
||||
ESCAPE = / [bfnrtv$\n\\'"] | x[a-fA-F0-9]{1,2} | [0-7]{1,3} /x # :nodoc:
|
||||
UNICODE_ESCAPE = / u[a-fA-F0-9]{4} /x # :nodoc: no 4-byte unicode chars? U[a-fA-F0-9]{8}
|
||||
REGEXP_ESCAPE = / [bfnrtv\n\\'"] | x[a-fA-F0-9]{1,2} | [0-7]{1,3} | \d | [bBdDsSwW\/] /x # :nodoc:
|
||||
|
||||
# TODO: interpretation inside ', ", /
|
||||
STRING_CONTENT_PATTERN = {
|
||||
"'" => /(?>\\[^\\'\n]+|[^\\'\n]+)+/,
|
||||
'"' => /[^\\$"\n]+/,
|
||||
"'''" => /(?>[^\\']+|'(?!''))+/,
|
||||
'"""' => /(?>[^\\$"]+|"(?!""))+/,
|
||||
'/' => /[^\\$\/\n]+/,
|
||||
} # :nodoc:
|
||||
|
||||
protected
|
||||
|
||||
def scan_tokens encoder, options
|
||||
|
||||
state = :initial
|
||||
inline_block_stack = []
|
||||
inline_block_paren_depth = nil
|
||||
string_delimiter = nil
|
||||
import_clause = class_name_follows = last_token = after_def = false
|
||||
value_expected = true
|
||||
|
||||
until eos?
|
||||
|
||||
case state
|
||||
|
||||
when :initial
|
||||
|
||||
if match = scan(/ \s+ | \\\n /x)
|
||||
encoder.text_token match, :space
|
||||
if match.index ?\n
|
||||
import_clause = after_def = false
|
||||
value_expected = true unless value_expected
|
||||
end
|
||||
next
|
||||
|
||||
elsif match = scan(%r! // [^\n\\]* (?: \\. [^\n\\]* )* | /\* (?: .*? \*/ | .* ) !mx)
|
||||
value_expected = true
|
||||
after_def = false
|
||||
encoder.text_token match, :comment
|
||||
|
||||
elsif bol? && match = scan(/ \#!.* /x)
|
||||
encoder.text_token match, :doctype
|
||||
|
||||
elsif import_clause && match = scan(/ (?!as) #{IDENT} (?: \. #{IDENT} )* (?: \.\* )? /ox)
|
||||
after_def = value_expected = false
|
||||
encoder.text_token match, :include
|
||||
|
||||
elsif match = scan(/ #{IDENT} | \[\] /ox)
|
||||
kind = IDENT_KIND[match]
|
||||
value_expected = (kind == :keyword) && KEYWORDS_EXPECTING_VALUE[match]
|
||||
if last_token == '.'
|
||||
kind = :ident
|
||||
elsif class_name_follows
|
||||
kind = :class
|
||||
class_name_follows = false
|
||||
elsif after_def && check(/\s*[({]/)
|
||||
kind = :method
|
||||
after_def = false
|
||||
elsif kind == :ident && last_token != '?' && check(/:/)
|
||||
kind = :key
|
||||
else
|
||||
class_name_follows = true if match == 'class' || (import_clause && match == 'as')
|
||||
import_clause = match == 'import'
|
||||
after_def = true if match == 'def'
|
||||
end
|
||||
encoder.text_token match, kind
|
||||
|
||||
elsif match = scan(/;/)
|
||||
import_clause = after_def = false
|
||||
value_expected = true
|
||||
encoder.text_token match, :operator
|
||||
|
||||
elsif match = scan(/\{/)
|
||||
class_name_follows = after_def = false
|
||||
value_expected = true
|
||||
encoder.text_token match, :operator
|
||||
if !inline_block_stack.empty?
|
||||
inline_block_paren_depth += 1
|
||||
end
|
||||
|
||||
# TODO: ~'...', ~"..." and ~/.../ style regexps
|
||||
elsif match = scan(/ \.\.<? | \*?\.(?!\d)@? | \.& | \?:? | [,?:(\[] | -[->] | \+\+ |
|
||||
&& | \|\| | \*\*=? | ==?~ | <=?>? | [-+*%^~&|>=!]=? | <<<?=? | >>>?=? /x)
|
||||
value_expected = true
|
||||
value_expected = :regexp if match == '~'
|
||||
after_def = false
|
||||
encoder.text_token match, :operator
|
||||
|
||||
elsif match = scan(/ [)\]}] /x)
|
||||
value_expected = after_def = false
|
||||
if !inline_block_stack.empty? && match == '}'
|
||||
inline_block_paren_depth -= 1
|
||||
if inline_block_paren_depth == 0 # closing brace of inline block reached
|
||||
encoder.text_token match, :inline_delimiter
|
||||
encoder.end_group :inline
|
||||
state, string_delimiter, inline_block_paren_depth = inline_block_stack.pop
|
||||
next
|
||||
end
|
||||
end
|
||||
encoder.text_token match, :operator
|
||||
|
||||
elsif check(/[\d.]/)
|
||||
after_def = value_expected = false
|
||||
if match = scan(/0[xX][0-9A-Fa-f]+/)
|
||||
encoder.text_token match, :hex
|
||||
elsif match = scan(/(?>0[0-7]+)(?![89.eEfF])/)
|
||||
encoder.text_token match, :octal
|
||||
elsif match = scan(/\d+[fFdD]|\d*\.\d+(?:[eE][+-]?\d+)?[fFdD]?|\d+[eE][+-]?\d+[fFdD]?/)
|
||||
encoder.text_token match, :float
|
||||
elsif match = scan(/\d+[lLgG]?/)
|
||||
encoder.text_token match, :integer
|
||||
end
|
||||
|
||||
elsif match = scan(/'''|"""/)
|
||||
after_def = value_expected = false
|
||||
state = :multiline_string
|
||||
encoder.begin_group :string
|
||||
string_delimiter = match
|
||||
encoder.text_token match, :delimiter
|
||||
|
||||
# TODO: record.'name' syntax
|
||||
elsif match = scan(/["']/)
|
||||
after_def = value_expected = false
|
||||
state = match == '/' ? :regexp : :string
|
||||
encoder.begin_group state
|
||||
string_delimiter = match
|
||||
encoder.text_token match, :delimiter
|
||||
|
||||
elsif value_expected && match = scan(/\//)
|
||||
after_def = value_expected = false
|
||||
encoder.begin_group :regexp
|
||||
state = :regexp
|
||||
string_delimiter = '/'
|
||||
encoder.text_token match, :delimiter
|
||||
|
||||
elsif match = scan(/ @ #{IDENT} /ox)
|
||||
after_def = value_expected = false
|
||||
encoder.text_token match, :annotation
|
||||
|
||||
elsif match = scan(/\//)
|
||||
after_def = false
|
||||
value_expected = true
|
||||
encoder.text_token match, :operator
|
||||
|
||||
else
|
||||
encoder.text_token getch, :error
|
||||
|
||||
end
|
||||
|
||||
when :string, :regexp, :multiline_string
|
||||
if match = scan(STRING_CONTENT_PATTERN[string_delimiter])
|
||||
encoder.text_token match, :content
|
||||
|
||||
elsif match = scan(state == :multiline_string ? /'''|"""/ : /["'\/]/)
|
||||
encoder.text_token match, :delimiter
|
||||
if state == :regexp
|
||||
# TODO: regexp modifiers? s, m, x, i?
|
||||
modifiers = scan(/[ix]+/)
|
||||
encoder.text_token modifiers, :modifier if modifiers && !modifiers.empty?
|
||||
end
|
||||
state = :string if state == :multiline_string
|
||||
encoder.end_group state
|
||||
string_delimiter = nil
|
||||
after_def = value_expected = false
|
||||
state = :initial
|
||||
next
|
||||
|
||||
elsif (state == :string || state == :multiline_string) &&
|
||||
(match = scan(/ \\ (?: #{ESCAPE} | #{UNICODE_ESCAPE} ) /mox))
|
||||
if string_delimiter[0] == ?' && !(match == "\\\\" || match == "\\'")
|
||||
encoder.text_token match, :content
|
||||
else
|
||||
encoder.text_token match, :char
|
||||
end
|
||||
elsif state == :regexp && match = scan(/ \\ (?: #{REGEXP_ESCAPE} | #{UNICODE_ESCAPE} ) /mox)
|
||||
encoder.text_token match, :char
|
||||
|
||||
elsif match = scan(/ \$ #{IDENT} /mox)
|
||||
encoder.begin_group :inline
|
||||
encoder.text_token '$', :inline_delimiter
|
||||
match = match[1..-1]
|
||||
encoder.text_token match, IDENT_KIND[match]
|
||||
encoder.end_group :inline
|
||||
next
|
||||
elsif match = scan(/ \$ \{ /x)
|
||||
encoder.begin_group :inline
|
||||
encoder.text_token match, :inline_delimiter
|
||||
inline_block_stack << [state, string_delimiter, inline_block_paren_depth]
|
||||
inline_block_paren_depth = 1
|
||||
state = :initial
|
||||
next
|
||||
|
||||
elsif match = scan(/ \$ /mx)
|
||||
encoder.text_token match, :content
|
||||
|
||||
elsif match = scan(/ \\. /mx)
|
||||
encoder.text_token match, :content # TODO: Shouldn't this be :error?
|
||||
|
||||
elsif match = scan(/ \\ | \n /x)
|
||||
encoder.end_group state
|
||||
encoder.text_token match, :error
|
||||
after_def = value_expected = false
|
||||
state = :initial
|
||||
|
||||
else
|
||||
raise_inspect "else case \" reached; %p not handled." % peek(1), encoder
|
||||
|
||||
end
|
||||
|
||||
else
|
||||
raise_inspect 'Unknown state', encoder
|
||||
|
||||
end
|
||||
|
||||
last_token = match unless [:space, :comment, :doctype].include? kind
|
||||
|
||||
end
|
||||
|
||||
if [:multiline_string, :string, :regexp].include? state
|
||||
encoder.end_group state
|
||||
end
|
||||
|
||||
encoder
|
||||
end
|
||||
|
||||
end
|
||||
|
||||
end
|
||||
end
|
||||
|
|
@ -0,0 +1,168 @@
|
|||
module CodeRay
|
||||
module Scanners
|
||||
|
||||
load :ruby
|
||||
load :html
|
||||
load :java_script
|
||||
|
||||
class HAML < Scanner
|
||||
|
||||
register_for :haml
|
||||
title 'HAML Template'
|
||||
|
||||
KINDS_NOT_LOC = HTML::KINDS_NOT_LOC
|
||||
|
||||
protected
|
||||
|
||||
def setup
|
||||
super
|
||||
@ruby_scanner = CodeRay.scanner :ruby, :tokens => @tokens, :keep_tokens => true
|
||||
@embedded_ruby_scanner = CodeRay.scanner :ruby, :tokens => @tokens, :keep_tokens => true, :state => @ruby_scanner.interpreted_string_state
|
||||
@html_scanner = CodeRay.scanner :html, :tokens => @tokens, :keep_tokens => true
|
||||
end
|
||||
|
||||
def scan_tokens encoder, options
|
||||
|
||||
match = nil
|
||||
code = ''
|
||||
|
||||
until eos?
|
||||
|
||||
if bol?
|
||||
if match = scan(/!!!.*/)
|
||||
encoder.text_token match, :doctype
|
||||
next
|
||||
end
|
||||
|
||||
if match = scan(/(?>( *)(\/(?!\[if)|-\#|:javascript|:ruby|:\w+) *)(?=\n)/)
|
||||
encoder.text_token match, :comment
|
||||
|
||||
code = self[2]
|
||||
if match = scan(/(?:\n+#{self[1]} .*)+/)
|
||||
case code
|
||||
when '/', '-#'
|
||||
encoder.text_token match, :comment
|
||||
when ':javascript'
|
||||
# TODO: recognize #{...} snippets inside JavaScript
|
||||
@java_script_scanner ||= CodeRay.scanner :java_script, :tokens => @tokens, :keep_tokens => true
|
||||
@java_script_scanner.tokenize match, :tokens => encoder
|
||||
when ':ruby'
|
||||
@ruby_scanner.tokenize match, :tokens => encoder
|
||||
when /:\w+/
|
||||
encoder.text_token match, :comment
|
||||
else
|
||||
raise 'else-case reached: %p' % [code]
|
||||
end
|
||||
end
|
||||
end
|
||||
|
||||
if match = scan(/ +/)
|
||||
encoder.text_token match, :space
|
||||
end
|
||||
|
||||
if match = scan(/\/.*/)
|
||||
encoder.text_token match, :comment
|
||||
next
|
||||
end
|
||||
|
||||
if match = scan(/\\/)
|
||||
encoder.text_token match, :plain
|
||||
if match = scan(/.+/)
|
||||
@html_scanner.tokenize match, :tokens => encoder
|
||||
end
|
||||
next
|
||||
end
|
||||
|
||||
tag = false
|
||||
|
||||
if match = scan(/%[\w:]+\/?/)
|
||||
encoder.text_token match, :tag
|
||||
# if match = scan(/( +)(.+)/)
|
||||
# encoder.text_token self[1], :space
|
||||
# @embedded_ruby_scanner.tokenize self[2], :tokens => encoder
|
||||
# end
|
||||
tag = true
|
||||
end
|
||||
|
||||
while match = scan(/([.#])[-\w]*\w/)
|
||||
encoder.text_token match, self[1] == '#' ? :constant : :class
|
||||
tag = true
|
||||
end
|
||||
|
||||
if tag && match = scan(/(\()([^)]+)?(\))?/)
|
||||
# TODO: recognize title=@title, class="widget_#{@widget.number}"
|
||||
encoder.text_token self[1], :plain
|
||||
@html_scanner.tokenize self[2], :tokens => encoder, :state => :attribute if self[2]
|
||||
encoder.text_token self[3], :plain if self[3]
|
||||
end
|
||||
|
||||
if tag && match = scan(/\{/)
|
||||
encoder.text_token match, :plain
|
||||
|
||||
code = ''
|
||||
level = 1
|
||||
while true
|
||||
code << scan(/([^\{\},\n]|, *\n?)*/)
|
||||
case match = getch
|
||||
when '{'
|
||||
level += 1
|
||||
code << match
|
||||
when '}'
|
||||
level -= 1
|
||||
if level > 0
|
||||
code << match
|
||||
else
|
||||
break
|
||||
end
|
||||
when "\n", ",", nil
|
||||
break
|
||||
end
|
||||
end
|
||||
@ruby_scanner.tokenize code, :tokens => encoder unless code.empty?
|
||||
|
||||
encoder.text_token match, :plain if match
|
||||
end
|
||||
|
||||
if tag && match = scan(/(\[)([^\]\n]+)?(\])?/)
|
||||
encoder.text_token self[1], :plain
|
||||
@ruby_scanner.tokenize self[2], :tokens => encoder if self[2]
|
||||
encoder.text_token self[3], :plain if self[3]
|
||||
end
|
||||
|
||||
if tag && match = scan(/\//)
|
||||
encoder.text_token match, :tag
|
||||
end
|
||||
|
||||
if scan(/(>?<?[-=]|[&!]=|(& |!)|~)( *)([^,\n\|]+(?:(, *|\|(?=.|\n.*\|$))\n?[^,\n\|]*)*)?/)
|
||||
encoder.text_token self[1] + self[3], :plain
|
||||
if self[4]
|
||||
if self[2]
|
||||
@embedded_ruby_scanner.tokenize self[4], :tokens => encoder
|
||||
else
|
||||
@ruby_scanner.tokenize self[4], :tokens => encoder
|
||||
end
|
||||
end
|
||||
elsif match = scan(/((?:<|><?)(?![!?\/\w]))?(.+)?/)
|
||||
encoder.text_token self[1], :plain if self[1]
|
||||
# TODO: recognize #{...} snippets
|
||||
@html_scanner.tokenize self[2], :tokens => encoder if self[2]
|
||||
end
|
||||
|
||||
elsif match = scan(/.+/)
|
||||
@html_scanner.tokenize match, :tokens => encoder
|
||||
|
||||
end
|
||||
|
||||
if match = scan(/\n/)
|
||||
encoder.text_token match, :space
|
||||
end
|
||||
end
|
||||
|
||||
encoder
|
||||
|
||||
end
|
||||
|
||||
end
|
||||
|
||||
end
|
||||
end
|
||||
|
|
@ -0,0 +1,250 @@
|
|||
module CodeRay
|
||||
module Scanners
|
||||
|
||||
# HTML Scanner
|
||||
#
|
||||
# Alias: +xhtml+
|
||||
#
|
||||
# See also: Scanners::XML
|
||||
class HTML < Scanner
|
||||
|
||||
register_for :html
|
||||
|
||||
KINDS_NOT_LOC = [
|
||||
:comment, :doctype, :preprocessor,
|
||||
:tag, :attribute_name, :operator,
|
||||
:attribute_value, :string,
|
||||
:plain, :entity, :error,
|
||||
] # :nodoc:
|
||||
|
||||
EVENT_ATTRIBUTES = %w(
|
||||
onabort onafterprint onbeforeprint onbeforeunload onblur oncanplay
|
||||
oncanplaythrough onchange onclick oncontextmenu oncuechange ondblclick
|
||||
ondrag ondragdrop ondragend ondragenter ondragleave ondragover
|
||||
ondragstart ondrop ondurationchange onemptied onended onerror onfocus
|
||||
onformchange onforminput onhashchange oninput oninvalid onkeydown
|
||||
onkeypress onkeyup onload onloadeddata onloadedmetadata onloadstart
|
||||
onmessage onmousedown onmousemove onmouseout onmouseover onmouseup
|
||||
onmousewheel onmove onoffline ononline onpagehide onpageshow onpause
|
||||
onplay onplaying onpopstate onprogress onratechange onreadystatechange
|
||||
onredo onreset onresize onscroll onseeked onseeking onselect onshow
|
||||
onstalled onstorage onsubmit onsuspend ontimeupdate onundo onunload
|
||||
onvolumechange onwaiting
|
||||
)
|
||||
|
||||
IN_ATTRIBUTE = WordList::CaseIgnoring.new(nil).
|
||||
add(EVENT_ATTRIBUTES, :script)
|
||||
|
||||
ATTR_NAME = /[\w.:-]+/ # :nodoc:
|
||||
TAG_END = /\/?>/ # :nodoc:
|
||||
HEX = /[0-9a-fA-F]/ # :nodoc:
|
||||
ENTITY = /
|
||||
&
|
||||
(?:
|
||||
\w+
|
||||
|
|
||||
\#
|
||||
(?:
|
||||
\d+
|
||||
|
|
||||
x#{HEX}+
|
||||
)
|
||||
)
|
||||
;
|
||||
/ox # :nodoc:
|
||||
|
||||
PLAIN_STRING_CONTENT = {
|
||||
"'" => /[^&'>\n]+/,
|
||||
'"' => /[^&">\n]+/,
|
||||
} # :nodoc:
|
||||
|
||||
def reset
|
||||
super
|
||||
@state = :initial
|
||||
@plain_string_content = nil
|
||||
end
|
||||
|
||||
protected
|
||||
|
||||
def setup
|
||||
@state = :initial
|
||||
@plain_string_content = nil
|
||||
end
|
||||
|
||||
def scan_java_script encoder, code
|
||||
if code && !code.empty?
|
||||
@java_script_scanner ||= Scanners::JavaScript.new '', :keep_tokens => true
|
||||
# encoder.begin_group :inline
|
||||
@java_script_scanner.tokenize code, :tokens => encoder
|
||||
# encoder.end_group :inline
|
||||
end
|
||||
end
|
||||
|
||||
def scan_tokens encoder, options
|
||||
state = options[:state] || @state
|
||||
plain_string_content = @plain_string_content
|
||||
in_tag = in_attribute = nil
|
||||
|
||||
encoder.begin_group :string if state == :attribute_value_string
|
||||
|
||||
until eos?
|
||||
|
||||
if state != :in_special_tag && match = scan(/\s+/m)
|
||||
encoder.text_token match, :space
|
||||
|
||||
else
|
||||
|
||||
case state
|
||||
|
||||
when :initial
|
||||
if match = scan(/<!--(?:.*?-->|.*)/m)
|
||||
encoder.text_token match, :comment
|
||||
elsif match = scan(/<!DOCTYPE(?:.*?>|.*)/m)
|
||||
encoder.text_token match, :doctype
|
||||
elsif match = scan(/<\?xml(?:.*?\?>|.*)/m)
|
||||
encoder.text_token match, :preprocessor
|
||||
elsif match = scan(/<\?(?:.*?\?>|.*)/m)
|
||||
encoder.text_token match, :comment
|
||||
elsif match = scan(/<\/[-\w.:]*>?/m)
|
||||
in_tag = nil
|
||||
encoder.text_token match, :tag
|
||||
elsif match = scan(/<(?:(script)|[-\w.:]+)(>)?/m)
|
||||
encoder.text_token match, :tag
|
||||
in_tag = self[1]
|
||||
if self[2]
|
||||
state = :in_special_tag if in_tag
|
||||
else
|
||||
state = :attribute
|
||||
end
|
||||
elsif match = scan(/[^<>&]+/)
|
||||
encoder.text_token match, :plain
|
||||
elsif match = scan(/#{ENTITY}/ox)
|
||||
encoder.text_token match, :entity
|
||||
elsif match = scan(/[<>&]/)
|
||||
in_tag = nil
|
||||
encoder.text_token match, :error
|
||||
else
|
||||
raise_inspect '[BUG] else-case reached with state %p' % [state], encoder
|
||||
end
|
||||
|
||||
when :attribute
|
||||
if match = scan(/#{TAG_END}/o)
|
||||
encoder.text_token match, :tag
|
||||
in_attribute = nil
|
||||
if in_tag
|
||||
state = :in_special_tag
|
||||
else
|
||||
state = :initial
|
||||
end
|
||||
elsif match = scan(/#{ATTR_NAME}/o)
|
||||
in_attribute = IN_ATTRIBUTE[match]
|
||||
encoder.text_token match, :attribute_name
|
||||
state = :attribute_equal
|
||||
else
|
||||
in_tag = nil
|
||||
encoder.text_token getch, :error
|
||||
end
|
||||
|
||||
when :attribute_equal
|
||||
if match = scan(/=/) #/
|
||||
encoder.text_token match, :operator
|
||||
state = :attribute_value
|
||||
else
|
||||
state = :attribute
|
||||
next
|
||||
end
|
||||
|
||||
when :attribute_value
|
||||
if match = scan(/#{ATTR_NAME}/o)
|
||||
encoder.text_token match, :attribute_value
|
||||
state = :attribute
|
||||
elsif match = scan(/["']/)
|
||||
if in_attribute == :script
|
||||
encoder.begin_group :inline
|
||||
encoder.text_token match, :inline_delimiter
|
||||
if scan(/javascript:[ \t]*/)
|
||||
encoder.text_token matched, :comment
|
||||
end
|
||||
code = scan_until(match == '"' ? /(?="|\z)/ : /(?='|\z)/)
|
||||
scan_java_script encoder, code
|
||||
match = scan(/["']/)
|
||||
encoder.text_token match, :inline_delimiter if match
|
||||
encoder.end_group :inline
|
||||
state = :attribute
|
||||
in_attribute = nil
|
||||
else
|
||||
encoder.begin_group :string
|
||||
state = :attribute_value_string
|
||||
plain_string_content = PLAIN_STRING_CONTENT[match]
|
||||
encoder.text_token match, :delimiter
|
||||
end
|
||||
elsif match = scan(/#{TAG_END}/o)
|
||||
encoder.text_token match, :tag
|
||||
state = :initial
|
||||
else
|
||||
encoder.text_token getch, :error
|
||||
end
|
||||
|
||||
when :attribute_value_string
|
||||
if match = scan(plain_string_content)
|
||||
encoder.text_token match, :content
|
||||
elsif match = scan(/['"]/)
|
||||
encoder.text_token match, :delimiter
|
||||
encoder.end_group :string
|
||||
state = :attribute
|
||||
elsif match = scan(/#{ENTITY}/ox)
|
||||
encoder.text_token match, :entity
|
||||
elsif match = scan(/&/)
|
||||
encoder.text_token match, :content
|
||||
elsif match = scan(/[\n>]/)
|
||||
encoder.end_group :string
|
||||
state = :initial
|
||||
encoder.text_token match, :error
|
||||
end
|
||||
|
||||
when :in_special_tag
|
||||
case in_tag
|
||||
when 'script'
|
||||
encoder.text_token match, :space if match = scan(/[ \t]*\n/)
|
||||
if scan(/(\s*<!--)(?:(.*?)(-->)|(.*))/m)
|
||||
code = self[2] || self[4]
|
||||
closing = self[3]
|
||||
encoder.text_token self[1], :comment
|
||||
else
|
||||
code = scan_until(/(?=(?:\n\s*)?<\/script>)|\z/)
|
||||
closing = false
|
||||
end
|
||||
unless code.empty?
|
||||
encoder.begin_group :inline
|
||||
scan_java_script encoder, code
|
||||
encoder.end_group :inline
|
||||
end
|
||||
encoder.text_token closing, :comment if closing
|
||||
state = :initial
|
||||
else
|
||||
raise 'unknown special tag: %p' % [in_tag]
|
||||
end
|
||||
|
||||
else
|
||||
raise_inspect 'Unknown state: %p' % [state], encoder
|
||||
|
||||
end
|
||||
|
||||
end
|
||||
|
||||
end
|
||||
|
||||
if options[:keep_state]
|
||||
@state = state
|
||||
@plain_string_content = plain_string_content
|
||||
end
|
||||
|
||||
encoder.end_group :string if state == :attribute_value_string
|
||||
|
||||
encoder
|
||||
end
|
||||
|
||||
end
|
||||
|
||||
end
|
||||
end
|
||||
|
|
@ -0,0 +1,174 @@
|
|||
module CodeRay
|
||||
module Scanners
|
||||
|
||||
# Scanner for Java.
|
||||
class Java < Scanner
|
||||
|
||||
register_for :java
|
||||
|
||||
autoload :BuiltinTypes, CodeRay.coderay_path('scanners', 'java', 'builtin_types')
|
||||
|
||||
# http://java.sun.com/docs/books/tutorial/java/nutsandbolts/_keywords.html
|
||||
KEYWORDS = %w[
|
||||
assert break case catch continue default do else
|
||||
finally for if instanceof import new package
|
||||
return switch throw try typeof while
|
||||
debugger export
|
||||
] # :nodoc:
|
||||
RESERVED = %w[ const goto ] # :nodoc:
|
||||
CONSTANTS = %w[ false null true ] # :nodoc:
|
||||
MAGIC_VARIABLES = %w[ this super ] # :nodoc:
|
||||
TYPES = %w[
|
||||
boolean byte char class double enum float int interface long
|
||||
short void
|
||||
] << '[]' # :nodoc: because int[] should be highlighted as a type
|
||||
DIRECTIVES = %w[
|
||||
abstract extends final implements native private protected public
|
||||
static strictfp synchronized throws transient volatile
|
||||
] # :nodoc:
|
||||
|
||||
IDENT_KIND = WordList.new(:ident).
|
||||
add(KEYWORDS, :keyword).
|
||||
add(RESERVED, :reserved).
|
||||
add(CONSTANTS, :predefined_constant).
|
||||
add(MAGIC_VARIABLES, :local_variable).
|
||||
add(TYPES, :type).
|
||||
add(BuiltinTypes::List, :predefined_type).
|
||||
add(BuiltinTypes::List.select { |builtin| builtin[/(Error|Exception)$/] }, :exception).
|
||||
add(DIRECTIVES, :directive) # :nodoc:
|
||||
|
||||
ESCAPE = / [bfnrtv\n\\'"] | x[a-fA-F0-9]{1,2} | [0-7]{1,3} /x # :nodoc:
|
||||
UNICODE_ESCAPE = / u[a-fA-F0-9]{4} | U[a-fA-F0-9]{8} /x # :nodoc:
|
||||
STRING_CONTENT_PATTERN = {
|
||||
"'" => /[^\\']+/,
|
||||
'"' => /[^\\"]+/,
|
||||
'/' => /[^\\\/]+/,
|
||||
} # :nodoc:
|
||||
IDENT = /[a-zA-Z_][A-Za-z_0-9]*/ # :nodoc:
|
||||
|
||||
protected
|
||||
|
||||
def scan_tokens encoder, options
|
||||
|
||||
state = :initial
|
||||
string_delimiter = nil
|
||||
package_name_expected = false
|
||||
class_name_follows = false
|
||||
last_token_dot = false
|
||||
|
||||
until eos?
|
||||
|
||||
case state
|
||||
|
||||
when :initial
|
||||
|
||||
if match = scan(/ \s+ | \\\n /x)
|
||||
encoder.text_token match, :space
|
||||
next
|
||||
|
||||
elsif match = scan(%r! // [^\n\\]* (?: \\. [^\n\\]* )* | /\* (?: .*? \*/ | .* ) !mx)
|
||||
encoder.text_token match, :comment
|
||||
next
|
||||
|
||||
elsif package_name_expected && match = scan(/ #{IDENT} (?: \. #{IDENT} )* /ox)
|
||||
encoder.text_token match, package_name_expected
|
||||
|
||||
elsif match = scan(/ #{IDENT} | \[\] /ox)
|
||||
kind = IDENT_KIND[match]
|
||||
if last_token_dot
|
||||
kind = :ident
|
||||
elsif class_name_follows
|
||||
kind = :class
|
||||
class_name_follows = false
|
||||
else
|
||||
case match
|
||||
when 'import'
|
||||
package_name_expected = :include
|
||||
when 'package'
|
||||
package_name_expected = :namespace
|
||||
when 'class', 'interface'
|
||||
class_name_follows = true
|
||||
end
|
||||
end
|
||||
encoder.text_token match, kind
|
||||
|
||||
elsif match = scan(/ \.(?!\d) | [,?:()\[\]}] | -- | \+\+ | && | \|\| | \*\*=? | [-+*\/%^~&|<>=!]=? | <<<?=? | >>>?=? /x)
|
||||
encoder.text_token match, :operator
|
||||
|
||||
elsif match = scan(/;/)
|
||||
package_name_expected = false
|
||||
encoder.text_token match, :operator
|
||||
|
||||
elsif match = scan(/\{/)
|
||||
class_name_follows = false
|
||||
encoder.text_token match, :operator
|
||||
|
||||
elsif check(/[\d.]/)
|
||||
if match = scan(/0[xX][0-9A-Fa-f]+/)
|
||||
encoder.text_token match, :hex
|
||||
elsif match = scan(/(?>0[0-7]+)(?![89.eEfF])/)
|
||||
encoder.text_token match, :octal
|
||||
elsif match = scan(/\d+[fFdD]|\d*\.\d+(?:[eE][+-]?\d+)?[fFdD]?|\d+[eE][+-]?\d+[fFdD]?/)
|
||||
encoder.text_token match, :float
|
||||
elsif match = scan(/\d+[lL]?/)
|
||||
encoder.text_token match, :integer
|
||||
end
|
||||
|
||||
elsif match = scan(/["']/)
|
||||
state = :string
|
||||
encoder.begin_group state
|
||||
string_delimiter = match
|
||||
encoder.text_token match, :delimiter
|
||||
|
||||
elsif match = scan(/ @ #{IDENT} /ox)
|
||||
encoder.text_token match, :annotation
|
||||
|
||||
else
|
||||
encoder.text_token getch, :error
|
||||
|
||||
end
|
||||
|
||||
when :string
|
||||
if match = scan(STRING_CONTENT_PATTERN[string_delimiter])
|
||||
encoder.text_token match, :content
|
||||
elsif match = scan(/["'\/]/)
|
||||
encoder.text_token match, :delimiter
|
||||
encoder.end_group state
|
||||
state = :initial
|
||||
string_delimiter = nil
|
||||
elsif state == :string && (match = scan(/ \\ (?: #{ESCAPE} | #{UNICODE_ESCAPE} ) /mox))
|
||||
if string_delimiter == "'" && !(match == "\\\\" || match == "\\'")
|
||||
encoder.text_token match, :content
|
||||
else
|
||||
encoder.text_token match, :char
|
||||
end
|
||||
elsif match = scan(/\\./m)
|
||||
encoder.text_token match, :content
|
||||
elsif match = scan(/ \\ | $ /x)
|
||||
encoder.end_group state
|
||||
state = :initial
|
||||
encoder.text_token match, :error
|
||||
else
|
||||
raise_inspect "else case \" reached; %p not handled." % peek(1), encoder
|
||||
end
|
||||
|
||||
else
|
||||
raise_inspect 'Unknown state', encoder
|
||||
|
||||
end
|
||||
|
||||
last_token_dot = match == '.'
|
||||
|
||||
end
|
||||
|
||||
if state == :string
|
||||
encoder.end_group state
|
||||
end
|
||||
|
||||
encoder
|
||||
end
|
||||
|
||||
end
|
||||
|
||||
end
|
||||
end
|
||||
|
|
@ -0,0 +1,421 @@
|
|||
module CodeRay
|
||||
module Scanners
|
||||
|
||||
module Java::BuiltinTypes # :nodoc:
|
||||
|
||||
#:nocov:
|
||||
List = %w[
|
||||
AbstractAction AbstractBorder AbstractButton AbstractCellEditor AbstractCollection
|
||||
AbstractColorChooserPanel AbstractDocument AbstractExecutorService AbstractInterruptibleChannel
|
||||
AbstractLayoutCache AbstractList AbstractListModel AbstractMap AbstractMethodError AbstractPreferences
|
||||
AbstractQueue AbstractQueuedSynchronizer AbstractSelectableChannel AbstractSelectionKey AbstractSelector
|
||||
AbstractSequentialList AbstractSet AbstractSpinnerModel AbstractTableModel AbstractUndoableEdit
|
||||
AbstractWriter AccessControlContext AccessControlException AccessController AccessException Accessible
|
||||
AccessibleAction AccessibleAttributeSequence AccessibleBundle AccessibleComponent AccessibleContext
|
||||
AccessibleEditableText AccessibleExtendedComponent AccessibleExtendedTable AccessibleExtendedText
|
||||
AccessibleHyperlink AccessibleHypertext AccessibleIcon AccessibleKeyBinding AccessibleObject
|
||||
AccessibleRelation AccessibleRelationSet AccessibleResourceBundle AccessibleRole AccessibleSelection
|
||||
AccessibleState AccessibleStateSet AccessibleStreamable AccessibleTable AccessibleTableModelChange
|
||||
AccessibleText AccessibleTextSequence AccessibleValue AccountException AccountExpiredException
|
||||
AccountLockedException AccountNotFoundException Acl AclEntry AclNotFoundException Action ActionEvent
|
||||
ActionListener ActionMap ActionMapUIResource Activatable ActivateFailedException ActivationDesc
|
||||
ActivationException ActivationGroup ActivationGroupDesc ActivationGroupID ActivationGroup_Stub
|
||||
ActivationID ActivationInstantiator ActivationMonitor ActivationSystem Activator ActiveEvent
|
||||
ActivityCompletedException ActivityRequiredException Adjustable AdjustmentEvent AdjustmentListener
|
||||
Adler32 AffineTransform AffineTransformOp AlgorithmParameterGenerator AlgorithmParameterGeneratorSpi
|
||||
AlgorithmParameters AlgorithmParameterSpec AlgorithmParametersSpi AllPermission AlphaComposite
|
||||
AlreadyBoundException AlreadyConnectedException AncestorEvent AncestorListener AnnotatedElement
|
||||
Annotation AnnotationFormatError AnnotationTypeMismatchException AppConfigurationEntry Appendable Applet
|
||||
AppletContext AppletInitializer AppletStub Arc2D Area AreaAveragingScaleFilter ArithmeticException Array
|
||||
ArrayBlockingQueue ArrayIndexOutOfBoundsException ArrayList Arrays ArrayStoreException ArrayType
|
||||
AssertionError AsyncBoxView AsynchronousCloseException AtomicBoolean AtomicInteger AtomicIntegerArray
|
||||
AtomicIntegerFieldUpdater AtomicLong AtomicLongArray AtomicLongFieldUpdater AtomicMarkableReference
|
||||
AtomicReference AtomicReferenceArray AtomicReferenceFieldUpdater AtomicStampedReference Attribute
|
||||
AttributeChangeNotification AttributeChangeNotificationFilter AttributedCharacterIterator
|
||||
AttributedString AttributeException AttributeInUseException AttributeList AttributeModificationException
|
||||
AttributeNotFoundException Attributes AttributeSet AttributeSetUtilities AttributeValueExp AudioClip
|
||||
AudioFileFormat AudioFileReader AudioFileWriter AudioFormat AudioInputStream AudioPermission AudioSystem
|
||||
AuthenticationException AuthenticationNotSupportedException Authenticator AuthorizeCallback
|
||||
AuthPermission AuthProvider Autoscroll AWTError AWTEvent AWTEventListener AWTEventListenerProxy
|
||||
AWTEventMulticaster AWTException AWTKeyStroke AWTPermission BackingStoreException
|
||||
BadAttributeValueExpException BadBinaryOpValueExpException BadLocationException BadPaddingException
|
||||
BadStringOperationException BandCombineOp BandedSampleModel BaseRowSet BasicArrowButton BasicAttribute
|
||||
BasicAttributes BasicBorders BasicButtonListener BasicButtonUI BasicCheckBoxMenuItemUI BasicCheckBoxUI
|
||||
BasicColorChooserUI BasicComboBoxEditor BasicComboBoxRenderer BasicComboBoxUI BasicComboPopup
|
||||
BasicControl BasicDesktopIconUI BasicDesktopPaneUI BasicDirectoryModel BasicEditorPaneUI
|
||||
BasicFileChooserUI BasicFormattedTextFieldUI BasicGraphicsUtils BasicHTML BasicIconFactory
|
||||
BasicInternalFrameTitlePane BasicInternalFrameUI BasicLabelUI BasicListUI BasicLookAndFeel
|
||||
BasicMenuBarUI BasicMenuItemUI BasicMenuUI BasicOptionPaneUI BasicPanelUI BasicPasswordFieldUI
|
||||
BasicPermission BasicPopupMenuSeparatorUI BasicPopupMenuUI BasicProgressBarUI BasicRadioButtonMenuItemUI
|
||||
BasicRadioButtonUI BasicRootPaneUI BasicScrollBarUI BasicScrollPaneUI BasicSeparatorUI BasicSliderUI
|
||||
BasicSpinnerUI BasicSplitPaneDivider BasicSplitPaneUI BasicStroke BasicTabbedPaneUI BasicTableHeaderUI
|
||||
BasicTableUI BasicTextAreaUI BasicTextFieldUI BasicTextPaneUI BasicTextUI BasicToggleButtonUI
|
||||
BasicToolBarSeparatorUI BasicToolBarUI BasicToolTipUI BasicTreeUI BasicViewportUI BatchUpdateException
|
||||
BeanContext BeanContextChild BeanContextChildComponentProxy BeanContextChildSupport
|
||||
BeanContextContainerProxy BeanContextEvent BeanContextMembershipEvent BeanContextMembershipListener
|
||||
BeanContextProxy BeanContextServiceAvailableEvent BeanContextServiceProvider
|
||||
BeanContextServiceProviderBeanInfo BeanContextServiceRevokedEvent BeanContextServiceRevokedListener
|
||||
BeanContextServices BeanContextServicesListener BeanContextServicesSupport BeanContextSupport
|
||||
BeanDescriptor BeanInfo Beans BevelBorder Bidi BigDecimal BigInteger BinaryRefAddr BindException Binding
|
||||
BitSet Blob BlockingQueue BlockView BMPImageWriteParam Book Boolean BooleanControl Border BorderFactory
|
||||
BorderLayout BorderUIResource BoundedRangeModel Box BoxLayout BoxView BreakIterator
|
||||
BrokenBarrierException Buffer BufferCapabilities BufferedImage BufferedImageFilter BufferedImageOp
|
||||
BufferedInputStream BufferedOutputStream BufferedReader BufferedWriter BufferOverflowException
|
||||
BufferStrategy BufferUnderflowException Button ButtonGroup ButtonModel ButtonUI Byte
|
||||
ByteArrayInputStream ByteArrayOutputStream ByteBuffer ByteChannel ByteLookupTable ByteOrder CachedRowSet
|
||||
CacheRequest CacheResponse Calendar Callable CallableStatement Callback CallbackHandler
|
||||
CancelablePrintJob CancellationException CancelledKeyException CannotProceedException
|
||||
CannotRedoException CannotUndoException Canvas CardLayout Caret CaretEvent CaretListener CellEditor
|
||||
CellEditorListener CellRendererPane Certificate CertificateEncodingException CertificateException
|
||||
CertificateExpiredException CertificateFactory CertificateFactorySpi CertificateNotYetValidException
|
||||
CertificateParsingException CertPath CertPathBuilder CertPathBuilderException CertPathBuilderResult
|
||||
CertPathBuilderSpi CertPathParameters CertPathTrustManagerParameters CertPathValidator
|
||||
CertPathValidatorException CertPathValidatorResult CertPathValidatorSpi CertSelector CertStore
|
||||
CertStoreException CertStoreParameters CertStoreSpi ChangedCharSetException ChangeEvent ChangeListener
|
||||
Channel Channels Character CharacterCodingException CharacterIterator CharArrayReader CharArrayWriter
|
||||
CharBuffer CharConversionException CharSequence Charset CharsetDecoder CharsetEncoder CharsetProvider
|
||||
Checkbox CheckboxGroup CheckboxMenuItem CheckedInputStream CheckedOutputStream Checksum Choice
|
||||
ChoiceCallback ChoiceFormat Chromaticity Cipher CipherInputStream CipherOutputStream CipherSpi Class
|
||||
ClassCastException ClassCircularityError ClassDefinition ClassDesc ClassFileTransformer ClassFormatError
|
||||
ClassLoader ClassLoaderRepository ClassLoadingMXBean ClassNotFoundException Clip Clipboard
|
||||
ClipboardOwner Clob Cloneable CloneNotSupportedException Closeable ClosedByInterruptException
|
||||
ClosedChannelException ClosedSelectorException CMMException CoderMalfunctionError CoderResult CodeSigner
|
||||
CodeSource CodingErrorAction CollationElementIterator CollationKey Collator Collection
|
||||
CollectionCertStoreParameters Collections Color ColorChooserComponentFactory ColorChooserUI
|
||||
ColorConvertOp ColorModel ColorSelectionModel ColorSpace ColorSupported ColorType ColorUIResource
|
||||
ComboBoxEditor ComboBoxModel ComboBoxUI ComboPopup CommunicationException Comparable Comparator
|
||||
CompilationMXBean Compiler CompletionService Component ComponentAdapter ComponentColorModel
|
||||
ComponentEvent ComponentInputMap ComponentInputMapUIResource ComponentListener ComponentOrientation
|
||||
ComponentSampleModel ComponentUI ComponentView Composite CompositeContext CompositeData
|
||||
CompositeDataSupport CompositeName CompositeType CompositeView CompoundBorder CompoundControl
|
||||
CompoundEdit CompoundName Compression ConcurrentHashMap ConcurrentLinkedQueue ConcurrentMap
|
||||
ConcurrentModificationException Condition Configuration ConfigurationException ConfirmationCallback
|
||||
ConnectException ConnectIOException Connection ConnectionEvent ConnectionEventListener
|
||||
ConnectionPendingException ConnectionPoolDataSource ConsoleHandler Constructor Container
|
||||
ContainerAdapter ContainerEvent ContainerListener ContainerOrderFocusTraversalPolicy ContentHandler
|
||||
ContentHandlerFactory ContentModel Context ContextNotEmptyException ContextualRenderedImageFactory
|
||||
Control ControlFactory ControllerEventListener ConvolveOp CookieHandler Copies CopiesSupported
|
||||
CopyOnWriteArrayList CopyOnWriteArraySet CountDownLatch CounterMonitor CounterMonitorMBean CRC32
|
||||
CredentialException CredentialExpiredException CredentialNotFoundException CRL CRLException CRLSelector
|
||||
CropImageFilter CSS CubicCurve2D Currency Cursor Customizer CyclicBarrier DatabaseMetaData DataBuffer
|
||||
DataBufferByte DataBufferDouble DataBufferFloat DataBufferInt DataBufferShort DataBufferUShort
|
||||
DataFlavor DataFormatException DatagramChannel DatagramPacket DatagramSocket DatagramSocketImpl
|
||||
DatagramSocketImplFactory DataInput DataInputStream DataLine DataOutput DataOutputStream DataSource
|
||||
DataTruncation DatatypeConfigurationException DatatypeConstants DatatypeFactory Date DateFormat
|
||||
DateFormatSymbols DateFormatter DateTimeAtCompleted DateTimeAtCreation DateTimeAtProcessing
|
||||
DateTimeSyntax DebugGraphics DecimalFormat DecimalFormatSymbols DefaultBoundedRangeModel
|
||||
DefaultButtonModel DefaultCaret DefaultCellEditor DefaultColorSelectionModel DefaultComboBoxModel
|
||||
DefaultDesktopManager DefaultEditorKit DefaultFocusManager DefaultFocusTraversalPolicy DefaultFormatter
|
||||
DefaultFormatterFactory DefaultHighlighter DefaultKeyboardFocusManager DefaultListCellRenderer
|
||||
DefaultListModel DefaultListSelectionModel DefaultLoaderRepository DefaultMenuLayout DefaultMetalTheme
|
||||
DefaultMutableTreeNode DefaultPersistenceDelegate DefaultSingleSelectionModel DefaultStyledDocument
|
||||
DefaultTableCellRenderer DefaultTableColumnModel DefaultTableModel DefaultTextUI DefaultTreeCellEditor
|
||||
DefaultTreeCellRenderer DefaultTreeModel DefaultTreeSelectionModel Deflater DeflaterOutputStream Delayed
|
||||
DelayQueue DelegationPermission Deprecated Descriptor DescriptorAccess DescriptorSupport DESedeKeySpec
|
||||
DesignMode DESKeySpec DesktopIconUI DesktopManager DesktopPaneUI Destination Destroyable
|
||||
DestroyFailedException DGC DHGenParameterSpec DHKey DHParameterSpec DHPrivateKey DHPrivateKeySpec
|
||||
DHPublicKey DHPublicKeySpec Dialog Dictionary DigestException DigestInputStream DigestOutputStream
|
||||
Dimension Dimension2D DimensionUIResource DirContext DirectColorModel DirectoryManager DirObjectFactory
|
||||
DirStateFactory DisplayMode DnDConstants Doc DocAttribute DocAttributeSet DocFlavor DocPrintJob Document
|
||||
DocumentBuilder DocumentBuilderFactory Documented DocumentEvent DocumentFilter DocumentListener
|
||||
DocumentName DocumentParser DomainCombiner DOMLocator DOMResult DOMSource Double DoubleBuffer
|
||||
DragGestureEvent DragGestureListener DragGestureRecognizer DragSource DragSourceAdapter
|
||||
DragSourceContext DragSourceDragEvent DragSourceDropEvent DragSourceEvent DragSourceListener
|
||||
DragSourceMotionListener Driver DriverManager DriverPropertyInfo DropTarget DropTargetAdapter
|
||||
DropTargetContext DropTargetDragEvent DropTargetDropEvent DropTargetEvent DropTargetListener DSAKey
|
||||
DSAKeyPairGenerator DSAParameterSpec DSAParams DSAPrivateKey DSAPrivateKeySpec DSAPublicKey
|
||||
DSAPublicKeySpec DTD DTDConstants DuplicateFormatFlagsException Duration DynamicMBean ECField ECFieldF2m
|
||||
ECFieldFp ECGenParameterSpec ECKey ECParameterSpec ECPoint ECPrivateKey ECPrivateKeySpec ECPublicKey
|
||||
ECPublicKeySpec EditorKit Element ElementIterator ElementType Ellipse2D EllipticCurve EmptyBorder
|
||||
EmptyStackException EncodedKeySpec Encoder EncryptedPrivateKeyInfo Entity Enum
|
||||
EnumConstantNotPresentException EnumControl Enumeration EnumMap EnumSet EnumSyntax EOFException Error
|
||||
ErrorListener ErrorManager EtchedBorder Event EventContext EventDirContext EventHandler EventListener
|
||||
EventListenerList EventListenerProxy EventObject EventQueue EventSetDescriptor Exception
|
||||
ExceptionInInitializerError ExceptionListener Exchanger ExecutionException Executor
|
||||
ExecutorCompletionService Executors ExecutorService ExemptionMechanism ExemptionMechanismException
|
||||
ExemptionMechanismSpi ExpandVetoException ExportException Expression ExtendedRequest ExtendedResponse
|
||||
Externalizable FactoryConfigurationError FailedLoginException FeatureDescriptor Fidelity Field
|
||||
FieldPosition FieldView File FileCacheImageInputStream FileCacheImageOutputStream FileChannel
|
||||
FileChooserUI FileDescriptor FileDialog FileFilter FileHandler FileImageInputStream
|
||||
FileImageOutputStream FileInputStream FileLock FileLockInterruptionException FilenameFilter FileNameMap
|
||||
FileNotFoundException FileOutputStream FilePermission FileReader FileSystemView FileView FileWriter
|
||||
Filter FilteredImageSource FilteredRowSet FilterInputStream FilterOutputStream FilterReader FilterWriter
|
||||
Finishings FixedHeightLayoutCache FlatteningPathIterator FlavorEvent FlavorException FlavorListener
|
||||
FlavorMap FlavorTable Float FloatBuffer FloatControl FlowLayout FlowView Flushable FocusAdapter
|
||||
FocusEvent FocusListener FocusManager FocusTraversalPolicy Font FontFormatException FontMetrics
|
||||
FontRenderContext FontUIResource Format FormatConversionProvider FormatFlagsConversionMismatchException
|
||||
Formattable FormattableFlags Formatter FormatterClosedException FormSubmitEvent FormView Frame Future
|
||||
FutureTask GapContent GarbageCollectorMXBean GatheringByteChannel GaugeMonitor GaugeMonitorMBean
|
||||
GeneralPath GeneralSecurityException GenericArrayType GenericDeclaration GenericSignatureFormatError
|
||||
GlyphJustificationInfo GlyphMetrics GlyphVector GlyphView GradientPaint GraphicAttribute Graphics
|
||||
Graphics2D GraphicsConfigTemplate GraphicsConfiguration GraphicsDevice GraphicsEnvironment GrayFilter
|
||||
GregorianCalendar GridBagConstraints GridBagLayout GridLayout Group Guard GuardedObject GZIPInputStream
|
||||
GZIPOutputStream Handler HandshakeCompletedEvent HandshakeCompletedListener HasControls HashAttributeSet
|
||||
HashDocAttributeSet HashMap HashPrintJobAttributeSet HashPrintRequestAttributeSet
|
||||
HashPrintServiceAttributeSet HashSet Hashtable HeadlessException HierarchyBoundsAdapter
|
||||
HierarchyBoundsListener HierarchyEvent HierarchyListener Highlighter HostnameVerifier HTML HTMLDocument
|
||||
HTMLEditorKit HTMLFrameHyperlinkEvent HTMLWriter HttpRetryException HttpsURLConnection HttpURLConnection
|
||||
HyperlinkEvent HyperlinkListener ICC_ColorSpace ICC_Profile ICC_ProfileGray ICC_ProfileRGB Icon
|
||||
IconUIResource IconView Identity IdentityHashMap IdentityScope IIOByteBuffer IIOException IIOImage
|
||||
IIOInvalidTreeException IIOMetadata IIOMetadataController IIOMetadataFormat IIOMetadataFormatImpl
|
||||
IIOMetadataNode IIOParam IIOParamController IIOReadProgressListener IIOReadUpdateListener
|
||||
IIOReadWarningListener IIORegistry IIOServiceProvider IIOWriteProgressListener IIOWriteWarningListener
|
||||
IllegalAccessError IllegalAccessException IllegalArgumentException IllegalBlockingModeException
|
||||
IllegalBlockSizeException IllegalCharsetNameException IllegalClassFormatException
|
||||
IllegalComponentStateException IllegalFormatCodePointException IllegalFormatConversionException
|
||||
IllegalFormatException IllegalFormatFlagsException IllegalFormatPrecisionException
|
||||
IllegalFormatWidthException IllegalMonitorStateException IllegalPathStateException
|
||||
IllegalSelectorException IllegalStateException IllegalThreadStateException Image ImageCapabilities
|
||||
ImageConsumer ImageFilter ImageGraphicAttribute ImageIcon ImageInputStream ImageInputStreamImpl
|
||||
ImageInputStreamSpi ImageIO ImageObserver ImageOutputStream ImageOutputStreamImpl ImageOutputStreamSpi
|
||||
ImageProducer ImageReader ImageReaderSpi ImageReaderWriterSpi ImageReadParam ImageTranscoder
|
||||
ImageTranscoderSpi ImageTypeSpecifier ImageView ImageWriteParam ImageWriter ImageWriterSpi
|
||||
ImagingOpException IncompatibleClassChangeError IncompleteAnnotationException IndexColorModel
|
||||
IndexedPropertyChangeEvent IndexedPropertyDescriptor IndexOutOfBoundsException Inet4Address Inet6Address
|
||||
InetAddress InetSocketAddress Inflater InflaterInputStream InheritableThreadLocal Inherited
|
||||
InitialContext InitialContextFactory InitialContextFactoryBuilder InitialDirContext InitialLdapContext
|
||||
InlineView InputContext InputEvent InputMap InputMapUIResource InputMethod InputMethodContext
|
||||
InputMethodDescriptor InputMethodEvent InputMethodHighlight InputMethodListener InputMethodRequests
|
||||
InputMismatchException InputStream InputStreamReader InputSubset InputVerifier Insets InsetsUIResource
|
||||
InstanceAlreadyExistsException InstanceNotFoundException InstantiationError InstantiationException
|
||||
Instrument Instrumentation InsufficientResourcesException IntBuffer Integer IntegerSyntax InternalError
|
||||
InternalFrameAdapter InternalFrameEvent InternalFrameFocusTraversalPolicy InternalFrameListener
|
||||
InternalFrameUI InternationalFormatter InterruptedException InterruptedIOException
|
||||
InterruptedNamingException InterruptibleChannel IntrospectionException Introspector
|
||||
InvalidActivityException InvalidAlgorithmParameterException InvalidApplicationException
|
||||
InvalidAttributeIdentifierException InvalidAttributesException InvalidAttributeValueException
|
||||
InvalidClassException InvalidDnDOperationException InvalidKeyException InvalidKeySpecException
|
||||
InvalidMarkException InvalidMidiDataException InvalidNameException InvalidObjectException
|
||||
InvalidOpenTypeException InvalidParameterException InvalidParameterSpecException
|
||||
InvalidPreferencesFormatException InvalidPropertiesFormatException InvalidRelationIdException
|
||||
InvalidRelationServiceException InvalidRelationTypeException InvalidRoleInfoException
|
||||
InvalidRoleValueException InvalidSearchControlsException InvalidSearchFilterException
|
||||
InvalidTargetObjectTypeException InvalidTransactionException InvocationEvent InvocationHandler
|
||||
InvocationTargetException IOException ItemEvent ItemListener ItemSelectable Iterable Iterator
|
||||
IvParameterSpec JApplet JarEntry JarException JarFile JarInputStream JarOutputStream JarURLConnection
|
||||
JButton JCheckBox JCheckBoxMenuItem JColorChooser JComboBox JComponent JdbcRowSet JDesktopPane JDialog
|
||||
JEditorPane JFileChooser JFormattedTextField JFrame JInternalFrame JLabel JLayeredPane JList JMenu
|
||||
JMenuBar JMenuItem JMException JMRuntimeException JMXAuthenticator JMXConnectionNotification
|
||||
JMXConnector JMXConnectorFactory JMXConnectorProvider JMXConnectorServer JMXConnectorServerFactory
|
||||
JMXConnectorServerMBean JMXConnectorServerProvider JMXPrincipal JMXProviderException
|
||||
JMXServerErrorException JMXServiceURL JobAttributes JobHoldUntil JobImpressions JobImpressionsCompleted
|
||||
JobImpressionsSupported JobKOctets JobKOctetsProcessed JobKOctetsSupported JobMediaSheets
|
||||
JobMediaSheetsCompleted JobMediaSheetsSupported JobMessageFromOperator JobName JobOriginatingUserName
|
||||
JobPriority JobPrioritySupported JobSheets JobState JobStateReason JobStateReasons Joinable JoinRowSet
|
||||
JOptionPane JPanel JPasswordField JPEGHuffmanTable JPEGImageReadParam JPEGImageWriteParam JPEGQTable
|
||||
JPopupMenu JProgressBar JRadioButton JRadioButtonMenuItem JRootPane JScrollBar JScrollPane JSeparator
|
||||
JSlider JSpinner JSplitPane JTabbedPane JTable JTableHeader JTextArea JTextComponent JTextField
|
||||
JTextPane JToggleButton JToolBar JToolTip JTree JViewport JWindow KerberosKey KerberosPrincipal
|
||||
KerberosTicket Kernel Key KeyAdapter KeyAgreement KeyAgreementSpi KeyAlreadyExistsException
|
||||
KeyboardFocusManager KeyEvent KeyEventDispatcher KeyEventPostProcessor KeyException KeyFactory
|
||||
KeyFactorySpi KeyGenerator KeyGeneratorSpi KeyListener KeyManagementException KeyManager
|
||||
KeyManagerFactory KeyManagerFactorySpi Keymap KeyPair KeyPairGenerator KeyPairGeneratorSpi KeyRep
|
||||
KeySpec KeyStore KeyStoreBuilderParameters KeyStoreException KeyStoreSpi KeyStroke Label LabelUI
|
||||
LabelView LanguageCallback LastOwnerException LayeredHighlighter LayoutFocusTraversalPolicy
|
||||
LayoutManager LayoutManager2 LayoutQueue LDAPCertStoreParameters LdapContext LdapName
|
||||
LdapReferralException Lease Level LimitExceededException Line Line2D LineBorder LineBreakMeasurer
|
||||
LineEvent LineListener LineMetrics LineNumberInputStream LineNumberReader LineUnavailableException
|
||||
LinkageError LinkedBlockingQueue LinkedHashMap LinkedHashSet LinkedList LinkException LinkLoopException
|
||||
LinkRef List ListCellRenderer ListDataEvent ListDataListener ListenerNotFoundException ListIterator
|
||||
ListModel ListResourceBundle ListSelectionEvent ListSelectionListener ListSelectionModel ListUI ListView
|
||||
LoaderHandler Locale LocateRegistry Lock LockSupport Logger LoggingMXBean LoggingPermission LoginContext
|
||||
LoginException LoginModule LogManager LogRecord LogStream Long LongBuffer LookAndFeel LookupOp
|
||||
LookupTable Mac MacSpi MalformedInputException MalformedLinkException MalformedObjectNameException
|
||||
MalformedParameterizedTypeException MalformedURLException ManagementFactory ManagementPermission
|
||||
ManageReferralControl ManagerFactoryParameters Manifest Map MappedByteBuffer MarshalException
|
||||
MarshalledObject MaskFormatter Matcher MatchResult Math MathContext MatteBorder MBeanAttributeInfo
|
||||
MBeanConstructorInfo MBeanException MBeanFeatureInfo MBeanInfo MBeanNotificationInfo MBeanOperationInfo
|
||||
MBeanParameterInfo MBeanPermission MBeanRegistration MBeanRegistrationException MBeanServer
|
||||
MBeanServerBuilder MBeanServerConnection MBeanServerDelegate MBeanServerDelegateMBean MBeanServerFactory
|
||||
MBeanServerForwarder MBeanServerInvocationHandler MBeanServerNotification MBeanServerNotificationFilter
|
||||
MBeanServerPermission MBeanTrustPermission Media MediaName MediaPrintableArea MediaSize MediaSizeName
|
||||
MediaTracker MediaTray Member MemoryCacheImageInputStream MemoryCacheImageOutputStream MemoryHandler
|
||||
MemoryImageSource MemoryManagerMXBean MemoryMXBean MemoryNotificationInfo MemoryPoolMXBean MemoryType
|
||||
MemoryUsage Menu MenuBar MenuBarUI MenuComponent MenuContainer MenuDragMouseEvent MenuDragMouseListener
|
||||
MenuElement MenuEvent MenuItem MenuItemUI MenuKeyEvent MenuKeyListener MenuListener MenuSelectionManager
|
||||
MenuShortcut MessageDigest MessageDigestSpi MessageFormat MetaEventListener MetalBorders MetalButtonUI
|
||||
MetalCheckBoxIcon MetalCheckBoxUI MetalComboBoxButton MetalComboBoxEditor MetalComboBoxIcon
|
||||
MetalComboBoxUI MetalDesktopIconUI MetalFileChooserUI MetalIconFactory MetalInternalFrameTitlePane
|
||||
MetalInternalFrameUI MetalLabelUI MetalLookAndFeel MetalMenuBarUI MetalPopupMenuSeparatorUI
|
||||
MetalProgressBarUI MetalRadioButtonUI MetalRootPaneUI MetalScrollBarUI MetalScrollButton
|
||||
MetalScrollPaneUI MetalSeparatorUI MetalSliderUI MetalSplitPaneUI MetalTabbedPaneUI MetalTextFieldUI
|
||||
MetalTheme MetalToggleButtonUI MetalToolBarUI MetalToolTipUI MetalTreeUI MetaMessage Method
|
||||
MethodDescriptor MGF1ParameterSpec MidiChannel MidiDevice MidiDeviceProvider MidiEvent MidiFileFormat
|
||||
MidiFileReader MidiFileWriter MidiMessage MidiSystem MidiUnavailableException MimeTypeParseException
|
||||
MinimalHTMLWriter MissingFormatArgumentException MissingFormatWidthException MissingResourceException
|
||||
Mixer MixerProvider MLet MLetMBean ModelMBean ModelMBeanAttributeInfo ModelMBeanConstructorInfo
|
||||
ModelMBeanInfo ModelMBeanInfoSupport ModelMBeanNotificationBroadcaster ModelMBeanNotificationInfo
|
||||
ModelMBeanOperationInfo ModificationItem Modifier Monitor MonitorMBean MonitorNotification
|
||||
MonitorSettingException MouseAdapter MouseDragGestureRecognizer MouseEvent MouseInfo MouseInputAdapter
|
||||
MouseInputListener MouseListener MouseMotionAdapter MouseMotionListener MouseWheelEvent
|
||||
MouseWheelListener MultiButtonUI MulticastSocket MultiColorChooserUI MultiComboBoxUI MultiDesktopIconUI
|
||||
MultiDesktopPaneUI MultiDoc MultiDocPrintJob MultiDocPrintService MultiFileChooserUI
|
||||
MultiInternalFrameUI MultiLabelUI MultiListUI MultiLookAndFeel MultiMenuBarUI MultiMenuItemUI
|
||||
MultiOptionPaneUI MultiPanelUI MultiPixelPackedSampleModel MultipleDocumentHandling MultipleMaster
|
||||
MultiPopupMenuUI MultiProgressBarUI MultiRootPaneUI MultiScrollBarUI MultiScrollPaneUI MultiSeparatorUI
|
||||
MultiSliderUI MultiSpinnerUI MultiSplitPaneUI MultiTabbedPaneUI MultiTableHeaderUI MultiTableUI
|
||||
MultiTextUI MultiToolBarUI MultiToolTipUI MultiTreeUI MultiViewportUI MutableAttributeSet
|
||||
MutableComboBoxModel MutableTreeNode Name NameAlreadyBoundException NameCallback NameClassPair
|
||||
NameNotFoundException NameParser NamespaceChangeListener NamespaceContext Naming NamingEnumeration
|
||||
NamingEvent NamingException NamingExceptionEvent NamingListener NamingManager NamingSecurityException
|
||||
NavigationFilter NegativeArraySizeException NetPermission NetworkInterface NoClassDefFoundError
|
||||
NoConnectionPendingException NodeChangeEvent NodeChangeListener NoInitialContextException
|
||||
NoninvertibleTransformException NonReadableChannelException NonWritableChannelException
|
||||
NoPermissionException NoRouteToHostException NoSuchAlgorithmException NoSuchAttributeException
|
||||
NoSuchElementException NoSuchFieldError NoSuchFieldException NoSuchMethodError NoSuchMethodException
|
||||
NoSuchObjectException NoSuchPaddingException NoSuchProviderException NotActiveException
|
||||
NotBoundException NotCompliantMBeanException NotContextException Notification NotificationBroadcaster
|
||||
NotificationBroadcasterSupport NotificationEmitter NotificationFilter NotificationFilterSupport
|
||||
NotificationListener NotificationResult NotOwnerException NotSerializableException NotYetBoundException
|
||||
NotYetConnectedException NullCipher NullPointerException Number NumberFormat NumberFormatException
|
||||
NumberFormatter NumberOfDocuments NumberOfInterveningJobs NumberUp NumberUpSupported NumericShaper
|
||||
OAEPParameterSpec Object ObjectChangeListener ObjectFactory ObjectFactoryBuilder ObjectInput
|
||||
ObjectInputStream ObjectInputValidation ObjectInstance ObjectName ObjectOutput ObjectOutputStream
|
||||
ObjectStreamClass ObjectStreamConstants ObjectStreamException ObjectStreamField ObjectView ObjID
|
||||
Observable Observer OceanTheme OpenDataException OpenMBeanAttributeInfo OpenMBeanAttributeInfoSupport
|
||||
OpenMBeanConstructorInfo OpenMBeanConstructorInfoSupport OpenMBeanInfo OpenMBeanInfoSupport
|
||||
OpenMBeanOperationInfo OpenMBeanOperationInfoSupport OpenMBeanParameterInfo
|
||||
OpenMBeanParameterInfoSupport OpenType OperatingSystemMXBean Operation OperationNotSupportedException
|
||||
OperationsException Option OptionalDataException OptionPaneUI OrientationRequested OutOfMemoryError
|
||||
OutputDeviceAssigned OutputKeys OutputStream OutputStreamWriter OverlappingFileLockException
|
||||
OverlayLayout Override Owner Pack200 Package PackedColorModel Pageable PageAttributes
|
||||
PagedResultsControl PagedResultsResponseControl PageFormat PageRanges PagesPerMinute PagesPerMinuteColor
|
||||
Paint PaintContext PaintEvent Panel PanelUI Paper ParagraphView ParameterBlock ParameterDescriptor
|
||||
ParameterizedType ParameterMetaData ParseException ParsePosition Parser ParserConfigurationException
|
||||
ParserDelegator PartialResultException PasswordAuthentication PasswordCallback PasswordView Patch
|
||||
PathIterator Pattern PatternSyntaxException PBEKey PBEKeySpec PBEParameterSpec PDLOverrideSupported
|
||||
Permission PermissionCollection Permissions PersistenceDelegate PersistentMBean PhantomReference Pipe
|
||||
PipedInputStream PipedOutputStream PipedReader PipedWriter PixelGrabber PixelInterleavedSampleModel
|
||||
PKCS8EncodedKeySpec PKIXBuilderParameters PKIXCertPathBuilderResult PKIXCertPathChecker
|
||||
PKIXCertPathValidatorResult PKIXParameters PlainDocument PlainView Point Point2D PointerInfo Policy
|
||||
PolicyNode PolicyQualifierInfo Polygon PooledConnection Popup PopupFactory PopupMenu PopupMenuEvent
|
||||
PopupMenuListener PopupMenuUI Port PortableRemoteObject PortableRemoteObjectDelegate
|
||||
PortUnreachableException Position Predicate PreferenceChangeEvent PreferenceChangeListener Preferences
|
||||
PreferencesFactory PreparedStatement PresentationDirection Principal Printable PrinterAbortException
|
||||
PrinterException PrinterGraphics PrinterInfo PrinterIOException PrinterIsAcceptingJobs PrinterJob
|
||||
PrinterLocation PrinterMakeAndModel PrinterMessageFromOperator PrinterMoreInfo
|
||||
PrinterMoreInfoManufacturer PrinterName PrinterResolution PrinterState PrinterStateReason
|
||||
PrinterStateReasons PrinterURI PrintEvent PrintException PrintGraphics PrintJob PrintJobAdapter
|
||||
PrintJobAttribute PrintJobAttributeEvent PrintJobAttributeListener PrintJobAttributeSet PrintJobEvent
|
||||
PrintJobListener PrintQuality PrintRequestAttribute PrintRequestAttributeSet PrintService
|
||||
PrintServiceAttribute PrintServiceAttributeEvent PrintServiceAttributeListener PrintServiceAttributeSet
|
||||
PrintServiceLookup PrintStream PrintWriter PriorityBlockingQueue PriorityQueue PrivateClassLoader
|
||||
PrivateCredentialPermission PrivateKey PrivateMLet PrivilegedAction PrivilegedActionException
|
||||
PrivilegedExceptionAction Process ProcessBuilder ProfileDataException ProgressBarUI ProgressMonitor
|
||||
ProgressMonitorInputStream Properties PropertyChangeEvent PropertyChangeListener
|
||||
PropertyChangeListenerProxy PropertyChangeSupport PropertyDescriptor PropertyEditor
|
||||
PropertyEditorManager PropertyEditorSupport PropertyPermission PropertyResourceBundle
|
||||
PropertyVetoException ProtectionDomain ProtocolException Provider ProviderException Proxy ProxySelector
|
||||
PSource PSSParameterSpec PublicKey PushbackInputStream PushbackReader QName QuadCurve2D Query QueryEval
|
||||
QueryExp Queue QueuedJobCount Random RandomAccess RandomAccessFile Raster RasterFormatException RasterOp
|
||||
RC2ParameterSpec RC5ParameterSpec Rdn Readable ReadableByteChannel Reader ReadOnlyBufferException
|
||||
ReadWriteLock RealmCallback RealmChoiceCallback Receiver Rectangle Rectangle2D RectangularShape
|
||||
ReentrantLock ReentrantReadWriteLock Ref RefAddr Reference Referenceable ReferenceQueue
|
||||
ReferenceUriSchemesSupported ReferralException ReflectionException ReflectPermission Refreshable
|
||||
RefreshFailedException Region RegisterableService Registry RegistryHandler RejectedExecutionException
|
||||
RejectedExecutionHandler Relation RelationException RelationNotFoundException RelationNotification
|
||||
RelationService RelationServiceMBean RelationServiceNotRegisteredException RelationSupport
|
||||
RelationSupportMBean RelationType RelationTypeNotFoundException RelationTypeSupport Remote RemoteCall
|
||||
RemoteException RemoteObject RemoteObjectInvocationHandler RemoteRef RemoteServer RemoteStub
|
||||
RenderableImage RenderableImageOp RenderableImageProducer RenderContext RenderedImage
|
||||
RenderedImageFactory Renderer RenderingHints RepaintManager ReplicateScaleFilter RequestingUserName
|
||||
RequiredModelMBean RescaleOp ResolutionSyntax Resolver ResolveResult ResourceBundle ResponseCache Result
|
||||
ResultSet ResultSetMetaData Retention RetentionPolicy ReverbType RGBImageFilter RMIClassLoader
|
||||
RMIClassLoaderSpi RMIClientSocketFactory RMIConnection RMIConnectionImpl RMIConnectionImpl_Stub
|
||||
RMIConnector RMIConnectorServer RMIFailureHandler RMIIIOPServerImpl RMIJRMPServerImpl
|
||||
RMISecurityException RMISecurityManager RMIServer RMIServerImpl RMIServerImpl_Stub
|
||||
RMIServerSocketFactory RMISocketFactory Robot Role RoleInfo RoleInfoNotFoundException RoleList
|
||||
RoleNotFoundException RoleResult RoleStatus RoleUnresolved RoleUnresolvedList RootPaneContainer
|
||||
RootPaneUI RoundingMode RoundRectangle2D RowMapper RowSet RowSetEvent RowSetInternal RowSetListener
|
||||
RowSetMetaData RowSetMetaDataImpl RowSetReader RowSetWarning RowSetWriter RSAKey RSAKeyGenParameterSpec
|
||||
RSAMultiPrimePrivateCrtKey RSAMultiPrimePrivateCrtKeySpec RSAOtherPrimeInfo RSAPrivateCrtKey
|
||||
RSAPrivateCrtKeySpec RSAPrivateKey RSAPrivateKeySpec RSAPublicKey RSAPublicKeySpec RTFEditorKit
|
||||
RuleBasedCollator Runnable Runtime RuntimeErrorException RuntimeException RuntimeMBeanException
|
||||
RuntimeMXBean RuntimeOperationsException RuntimePermission SampleModel Sasl SaslClient SaslClientFactory
|
||||
SaslException SaslServer SaslServerFactory Savepoint SAXParser SAXParserFactory SAXResult SAXSource
|
||||
SAXTransformerFactory Scanner ScatteringByteChannel ScheduledExecutorService ScheduledFuture
|
||||
ScheduledThreadPoolExecutor Schema SchemaFactory SchemaFactoryLoader SchemaViolationException Scrollable
|
||||
Scrollbar ScrollBarUI ScrollPane ScrollPaneAdjustable ScrollPaneConstants ScrollPaneLayout ScrollPaneUI
|
||||
SealedObject SearchControls SearchResult SecretKey SecretKeyFactory SecretKeyFactorySpi SecretKeySpec
|
||||
SecureCacheResponse SecureClassLoader SecureRandom SecureRandomSpi Security SecurityException
|
||||
SecurityManager SecurityPermission Segment SelectableChannel SelectionKey Selector SelectorProvider
|
||||
Semaphore SeparatorUI Sequence SequenceInputStream Sequencer SerialArray SerialBlob SerialClob
|
||||
SerialDatalink SerialException Serializable SerializablePermission SerialJavaObject SerialRef
|
||||
SerialStruct ServerCloneException ServerError ServerException ServerNotActiveException ServerRef
|
||||
ServerRuntimeException ServerSocket ServerSocketChannel ServerSocketFactory ServiceNotFoundException
|
||||
ServicePermission ServiceRegistry ServiceUI ServiceUIFactory ServiceUnavailableException Set
|
||||
SetOfIntegerSyntax Severity Shape ShapeGraphicAttribute SheetCollate Short ShortBuffer
|
||||
ShortBufferException ShortLookupTable ShortMessage Sides Signature SignatureException SignatureSpi
|
||||
SignedObject Signer SimpleAttributeSet SimpleBeanInfo SimpleDateFormat SimpleDoc SimpleFormatter
|
||||
SimpleTimeZone SimpleType SinglePixelPackedSampleModel SingleSelectionModel Size2DSyntax
|
||||
SizeLimitExceededException SizeRequirements SizeSequence Skeleton SkeletonMismatchException
|
||||
SkeletonNotFoundException SliderUI Socket SocketAddress SocketChannel SocketException SocketFactory
|
||||
SocketHandler SocketImpl SocketImplFactory SocketOptions SocketPermission SocketSecurityException
|
||||
SocketTimeoutException SoftBevelBorder SoftReference SortControl SortedMap SortedSet
|
||||
SortingFocusTraversalPolicy SortKey SortResponseControl Soundbank SoundbankReader SoundbankResource
|
||||
Source SourceDataLine SourceLocator SpinnerDateModel SpinnerListModel SpinnerModel SpinnerNumberModel
|
||||
SpinnerUI SplitPaneUI Spring SpringLayout SQLData SQLException SQLInput SQLInputImpl SQLOutput
|
||||
SQLOutputImpl SQLPermission SQLWarning SSLContext SSLContextSpi SSLEngine SSLEngineResult SSLException
|
||||
SSLHandshakeException SSLKeyException SSLPeerUnverifiedException SSLPermission SSLProtocolException
|
||||
SslRMIClientSocketFactory SslRMIServerSocketFactory SSLServerSocket SSLServerSocketFactory SSLSession
|
||||
SSLSessionBindingEvent SSLSessionBindingListener SSLSessionContext SSLSocket SSLSocketFactory Stack
|
||||
StackOverflowError StackTraceElement StandardMBean StartTlsRequest StartTlsResponse StateEdit
|
||||
StateEditable StateFactory Statement StreamCorruptedException StreamHandler StreamPrintService
|
||||
StreamPrintServiceFactory StreamResult StreamSource StreamTokenizer StrictMath String StringBuffer
|
||||
StringBufferInputStream StringBuilder StringCharacterIterator StringContent
|
||||
StringIndexOutOfBoundsException StringMonitor StringMonitorMBean StringReader StringRefAddr
|
||||
StringSelection StringTokenizer StringValueExp StringWriter Stroke Struct Stub StubDelegate
|
||||
StubNotFoundException Style StyleConstants StyleContext StyledDocument StyledEditorKit StyleSheet
|
||||
Subject SubjectDelegationPermission SubjectDomainCombiner SupportedValuesAttribute SuppressWarnings
|
||||
SwingConstants SwingPropertyChangeSupport SwingUtilities SyncFactory SyncFactoryException
|
||||
SyncFailedException SynchronousQueue SyncProvider SyncProviderException SyncResolver SynthConstants
|
||||
SynthContext Synthesizer SynthGraphicsUtils SynthLookAndFeel SynthPainter SynthStyle SynthStyleFactory
|
||||
SysexMessage System SystemColor SystemFlavorMap TabableView TabbedPaneUI TabExpander TableCellEditor
|
||||
TableCellRenderer TableColumn TableColumnModel TableColumnModelEvent TableColumnModelListener
|
||||
TableHeaderUI TableModel TableModelEvent TableModelListener TableUI TableView TabSet TabStop TabularData
|
||||
TabularDataSupport TabularType TagElement Target TargetDataLine TargetedNotification Templates
|
||||
TemplatesHandler TextAction TextArea TextAttribute TextComponent TextEvent TextField TextHitInfo
|
||||
TextInputCallback TextLayout TextListener TextMeasurer TextOutputCallback TextSyntax TextUI TexturePaint
|
||||
Thread ThreadDeath ThreadFactory ThreadGroup ThreadInfo ThreadLocal ThreadMXBean ThreadPoolExecutor
|
||||
Throwable Tie TileObserver Time TimeLimitExceededException TimeoutException Timer
|
||||
TimerAlarmClockNotification TimerMBean TimerNotification TimerTask Timestamp TimeUnit TimeZone
|
||||
TitledBorder ToolBarUI Toolkit ToolTipManager ToolTipUI TooManyListenersException Track
|
||||
TransactionalWriter TransactionRequiredException TransactionRolledbackException Transferable
|
||||
TransferHandler TransformAttribute Transformer TransformerConfigurationException TransformerException
|
||||
TransformerFactory TransformerFactoryConfigurationError TransformerHandler Transmitter Transparency
|
||||
TreeCellEditor TreeCellRenderer TreeExpansionEvent TreeExpansionListener TreeMap TreeModel
|
||||
TreeModelEvent TreeModelListener TreeNode TreePath TreeSelectionEvent TreeSelectionListener
|
||||
TreeSelectionModel TreeSet TreeUI TreeWillExpandListener TrustAnchor TrustManager TrustManagerFactory
|
||||
TrustManagerFactorySpi Type TypeInfoProvider TypeNotPresentException Types TypeVariable UID UIDefaults
|
||||
UIManager UIResource UndeclaredThrowableException UndoableEdit UndoableEditEvent UndoableEditListener
|
||||
UndoableEditSupport UndoManager UnexpectedException UnicastRemoteObject UnknownError
|
||||
UnknownFormatConversionException UnknownFormatFlagsException UnknownGroupException UnknownHostException
|
||||
UnknownObjectException UnknownServiceException UnmappableCharacterException UnmarshalException
|
||||
UnmodifiableClassException UnmodifiableSetException UnrecoverableEntryException
|
||||
UnrecoverableKeyException Unreferenced UnresolvedAddressException UnresolvedPermission
|
||||
UnsatisfiedLinkError UnsolicitedNotification UnsolicitedNotificationEvent
|
||||
UnsolicitedNotificationListener UnsupportedAddressTypeException UnsupportedAudioFileException
|
||||
UnsupportedCallbackException UnsupportedCharsetException UnsupportedClassVersionError
|
||||
UnsupportedEncodingException UnsupportedFlavorException UnsupportedLookAndFeelException
|
||||
UnsupportedOperationException URI URIException URIResolver URISyntax URISyntaxException URL
|
||||
URLClassLoader URLConnection URLDecoder URLEncoder URLStreamHandler URLStreamHandlerFactory
|
||||
UTFDataFormatException Util UtilDelegate Utilities UUID Validator ValidatorHandler ValueExp ValueHandler
|
||||
ValueHandlerMultiFormat VariableHeightLayoutCache Vector VerifyError VetoableChangeListener
|
||||
VetoableChangeListenerProxy VetoableChangeSupport View ViewFactory ViewportLayout ViewportUI
|
||||
VirtualMachineError Visibility VMID VoiceStatus Void VolatileImage WeakHashMap WeakReference WebRowSet
|
||||
WildcardType Window WindowAdapter WindowConstants WindowEvent WindowFocusListener WindowListener
|
||||
WindowStateListener WrappedPlainView WritableByteChannel WritableRaster WritableRenderedImage
|
||||
WriteAbortedException Writer X500Principal X500PrivateCredential X509Certificate X509CertSelector
|
||||
X509CRL X509CRLEntry X509CRLSelector X509EncodedKeySpec X509ExtendedKeyManager X509Extension
|
||||
X509KeyManager X509TrustManager XAConnection XADataSource XAException XAResource Xid XMLConstants
|
||||
XMLDecoder XMLEncoder XMLFormatter XMLGregorianCalendar XMLParseException XmlReader XmlWriter XPath
|
||||
XPathConstants XPathException XPathExpression XPathExpressionException XPathFactory
|
||||
XPathFactoryConfigurationException XPathFunction XPathFunctionException XPathFunctionResolver
|
||||
XPathVariableResolver ZipEntry ZipException ZipFile ZipInputStream ZipOutputStream ZoneView
|
||||
]
|
||||
#:nocov:
|
||||
|
||||
end
|
||||
|
||||
end
|
||||
end
|
||||
|
|
@ -0,0 +1,213 @@
|
|||
module CodeRay
|
||||
module Scanners
|
||||
|
||||
# Scanner for JavaScript.
|
||||
#
|
||||
# Aliases: +ecmascript+, +ecma_script+, +javascript+
|
||||
class JavaScript < Scanner
|
||||
|
||||
register_for :java_script
|
||||
file_extension 'js'
|
||||
|
||||
# The actual JavaScript keywords.
|
||||
KEYWORDS = %w[
|
||||
break case catch continue default delete do else
|
||||
finally for function if in instanceof new
|
||||
return switch throw try typeof var void while with
|
||||
] # :nodoc:
|
||||
PREDEFINED_CONSTANTS = %w[
|
||||
false null true undefined NaN Infinity
|
||||
] # :nodoc:
|
||||
|
||||
MAGIC_VARIABLES = %w[ this arguments ] # :nodoc: arguments was introduced in JavaScript 1.4
|
||||
|
||||
KEYWORDS_EXPECTING_VALUE = WordList.new.add %w[
|
||||
case delete in instanceof new return throw typeof with
|
||||
] # :nodoc:
|
||||
|
||||
# Reserved for future use.
|
||||
RESERVED_WORDS = %w[
|
||||
abstract boolean byte char class debugger double enum export extends
|
||||
final float goto implements import int interface long native package
|
||||
private protected public short static super synchronized throws transient
|
||||
volatile
|
||||
] # :nodoc:
|
||||
|
||||
IDENT_KIND = WordList.new(:ident).
|
||||
add(RESERVED_WORDS, :reserved).
|
||||
add(PREDEFINED_CONSTANTS, :predefined_constant).
|
||||
add(MAGIC_VARIABLES, :local_variable).
|
||||
add(KEYWORDS, :keyword) # :nodoc:
|
||||
|
||||
ESCAPE = / [bfnrtv\n\\'"] | x[a-fA-F0-9]{1,2} | [0-7]{1,3} /x # :nodoc:
|
||||
UNICODE_ESCAPE = / u[a-fA-F0-9]{4} | U[a-fA-F0-9]{8} /x # :nodoc:
|
||||
REGEXP_ESCAPE = / [bBdDsSwW] /x # :nodoc:
|
||||
STRING_CONTENT_PATTERN = {
|
||||
"'" => /[^\\']+/,
|
||||
'"' => /[^\\"]+/,
|
||||
'/' => /[^\\\/]+/,
|
||||
} # :nodoc:
|
||||
KEY_CHECK_PATTERN = {
|
||||
"'" => / (?> [^\\']* (?: \\. [^\\']* )* ) ' \s* : /mx,
|
||||
'"' => / (?> [^\\"]* (?: \\. [^\\"]* )* ) " \s* : /mx,
|
||||
} # :nodoc:
|
||||
|
||||
protected
|
||||
|
||||
def scan_tokens encoder, options
|
||||
|
||||
state = :initial
|
||||
string_delimiter = nil
|
||||
value_expected = true
|
||||
key_expected = false
|
||||
function_expected = false
|
||||
|
||||
until eos?
|
||||
|
||||
case state
|
||||
|
||||
when :initial
|
||||
|
||||
if match = scan(/ \s+ | \\\n /x)
|
||||
value_expected = true if !value_expected && match.index(?\n)
|
||||
encoder.text_token match, :space
|
||||
|
||||
elsif match = scan(%r! // [^\n\\]* (?: \\. [^\n\\]* )* | /\* (?: .*? \*/ | .* ) !mx)
|
||||
value_expected = true
|
||||
encoder.text_token match, :comment
|
||||
|
||||
elsif check(/\.?\d/)
|
||||
key_expected = value_expected = false
|
||||
if match = scan(/0[xX][0-9A-Fa-f]+/)
|
||||
encoder.text_token match, :hex
|
||||
elsif match = scan(/(?>0[0-7]+)(?![89.eEfF])/)
|
||||
encoder.text_token match, :octal
|
||||
elsif match = scan(/\d+[fF]|\d*\.\d+(?:[eE][+-]?\d+)?[fF]?|\d+[eE][+-]?\d+[fF]?/)
|
||||
encoder.text_token match, :float
|
||||
elsif match = scan(/\d+/)
|
||||
encoder.text_token match, :integer
|
||||
end
|
||||
|
||||
elsif value_expected && match = scan(/<([[:alpha:]]\w*) (?: [^\/>]*\/> | .*?<\/\1>)/xim)
|
||||
# TODO: scan over nested tags
|
||||
xml_scanner.tokenize match, :tokens => encoder
|
||||
value_expected = false
|
||||
next
|
||||
|
||||
elsif match = scan(/ [-+*=<>?:;,!&^|(\[{~%]+ | \.(?!\d) /x)
|
||||
value_expected = true
|
||||
last_operator = match[-1]
|
||||
key_expected = (last_operator == ?{) || (last_operator == ?,)
|
||||
function_expected = false
|
||||
encoder.text_token match, :operator
|
||||
|
||||
elsif match = scan(/ [)\]}]+ /x)
|
||||
function_expected = key_expected = value_expected = false
|
||||
encoder.text_token match, :operator
|
||||
|
||||
elsif match = scan(/ [$a-zA-Z_][A-Za-z_0-9$]* /x)
|
||||
kind = IDENT_KIND[match]
|
||||
value_expected = (kind == :keyword) && KEYWORDS_EXPECTING_VALUE[match]
|
||||
# TODO: labels
|
||||
if kind == :ident
|
||||
if match.index(?$) # $ allowed inside an identifier
|
||||
kind = :predefined
|
||||
elsif function_expected
|
||||
kind = :function
|
||||
elsif check(/\s*[=:]\s*function\b/)
|
||||
kind = :function
|
||||
elsif key_expected && check(/\s*:/)
|
||||
kind = :key
|
||||
end
|
||||
end
|
||||
function_expected = (kind == :keyword) && (match == 'function')
|
||||
key_expected = false
|
||||
encoder.text_token match, kind
|
||||
|
||||
elsif match = scan(/["']/)
|
||||
if key_expected && check(KEY_CHECK_PATTERN[match])
|
||||
state = :key
|
||||
else
|
||||
state = :string
|
||||
end
|
||||
encoder.begin_group state
|
||||
string_delimiter = match
|
||||
encoder.text_token match, :delimiter
|
||||
|
||||
elsif value_expected && (match = scan(/\//))
|
||||
encoder.begin_group :regexp
|
||||
state = :regexp
|
||||
string_delimiter = '/'
|
||||
encoder.text_token match, :delimiter
|
||||
|
||||
elsif match = scan(/ \/ /x)
|
||||
value_expected = true
|
||||
key_expected = false
|
||||
encoder.text_token match, :operator
|
||||
|
||||
else
|
||||
encoder.text_token getch, :error
|
||||
|
||||
end
|
||||
|
||||
when :string, :regexp, :key
|
||||
if match = scan(STRING_CONTENT_PATTERN[string_delimiter])
|
||||
encoder.text_token match, :content
|
||||
elsif match = scan(/["'\/]/)
|
||||
encoder.text_token match, :delimiter
|
||||
if state == :regexp
|
||||
modifiers = scan(/[gim]+/)
|
||||
encoder.text_token modifiers, :modifier if modifiers && !modifiers.empty?
|
||||
end
|
||||
encoder.end_group state
|
||||
string_delimiter = nil
|
||||
key_expected = value_expected = false
|
||||
state = :initial
|
||||
elsif state != :regexp && (match = scan(/ \\ (?: #{ESCAPE} | #{UNICODE_ESCAPE} ) /mox))
|
||||
if string_delimiter == "'" && !(match == "\\\\" || match == "\\'")
|
||||
encoder.text_token match, :content
|
||||
else
|
||||
encoder.text_token match, :char
|
||||
end
|
||||
elsif state == :regexp && match = scan(/ \\ (?: #{ESCAPE} | #{REGEXP_ESCAPE} | #{UNICODE_ESCAPE} ) /mox)
|
||||
encoder.text_token match, :char
|
||||
elsif match = scan(/\\./m)
|
||||
encoder.text_token match, :content
|
||||
elsif match = scan(/ \\ | $ /x)
|
||||
encoder.end_group state
|
||||
encoder.text_token match, :error
|
||||
key_expected = value_expected = false
|
||||
state = :initial
|
||||
else
|
||||
raise_inspect "else case \" reached; %p not handled." % peek(1), encoder
|
||||
end
|
||||
|
||||
else
|
||||
raise_inspect 'Unknown state', encoder
|
||||
|
||||
end
|
||||
|
||||
end
|
||||
|
||||
if [:string, :regexp].include? state
|
||||
encoder.end_group state
|
||||
end
|
||||
|
||||
encoder
|
||||
end
|
||||
|
||||
protected
|
||||
|
||||
def reset_instance
|
||||
super
|
||||
@xml_scanner.reset if defined? @xml_scanner
|
||||
end
|
||||
|
||||
def xml_scanner
|
||||
@xml_scanner ||= CodeRay.scanner :xml, :tokens => @tokens, :keep_tokens => true, :keep_state => false
|
||||
end
|
||||
|
||||
end
|
||||
|
||||
end
|
||||
end
|
||||
|
|
@ -0,0 +1,95 @@
|
|||
module CodeRay
|
||||
module Scanners
|
||||
|
||||
# Scanner for JSON (JavaScript Object Notation).
|
||||
class JSON < Scanner
|
||||
|
||||
register_for :json
|
||||
file_extension 'json'
|
||||
|
||||
KINDS_NOT_LOC = [
|
||||
:float, :char, :content, :delimiter,
|
||||
:error, :integer, :operator, :value,
|
||||
] # :nodoc:
|
||||
|
||||
ESCAPE = / [bfnrt\\"\/] /x # :nodoc:
|
||||
UNICODE_ESCAPE = / u[a-fA-F0-9]{4} /x # :nodoc:
|
||||
|
||||
protected
|
||||
|
||||
# See http://json.org/ for a definition of the JSON lexic/grammar.
|
||||
def scan_tokens encoder, options
|
||||
|
||||
state = :initial
|
||||
stack = []
|
||||
key_expected = false
|
||||
|
||||
until eos?
|
||||
|
||||
case state
|
||||
|
||||
when :initial
|
||||
if match = scan(/ \s+ /x)
|
||||
encoder.text_token match, :space
|
||||
elsif match = scan(/"/)
|
||||
state = key_expected ? :key : :string
|
||||
encoder.begin_group state
|
||||
encoder.text_token match, :delimiter
|
||||
elsif match = scan(/ [:,\[{\]}] /x)
|
||||
encoder.text_token match, :operator
|
||||
case match
|
||||
when ':' then key_expected = false
|
||||
when ',' then key_expected = true if stack.last == :object
|
||||
when '{' then stack << :object; key_expected = true
|
||||
when '[' then stack << :array
|
||||
when '}', ']' then stack.pop # no error recovery, but works for valid JSON
|
||||
end
|
||||
elsif match = scan(/ true | false | null /x)
|
||||
encoder.text_token match, :value
|
||||
elsif match = scan(/ -? (?: 0 | [1-9]\d* ) /x)
|
||||
if scan(/ \.\d+ (?:[eE][-+]?\d+)? | [eE][-+]? \d+ /x)
|
||||
match << matched
|
||||
encoder.text_token match, :float
|
||||
else
|
||||
encoder.text_token match, :integer
|
||||
end
|
||||
else
|
||||
encoder.text_token getch, :error
|
||||
end
|
||||
|
||||
when :string, :key
|
||||
if match = scan(/[^\\"]+/)
|
||||
encoder.text_token match, :content
|
||||
elsif match = scan(/"/)
|
||||
encoder.text_token match, :delimiter
|
||||
encoder.end_group state
|
||||
state = :initial
|
||||
elsif match = scan(/ \\ (?: #{ESCAPE} | #{UNICODE_ESCAPE} ) /mox)
|
||||
encoder.text_token match, :char
|
||||
elsif match = scan(/\\./m)
|
||||
encoder.text_token match, :content
|
||||
elsif match = scan(/ \\ | $ /x)
|
||||
encoder.end_group state
|
||||
encoder.text_token match, :error
|
||||
state = :initial
|
||||
else
|
||||
raise_inspect "else case \" reached; %p not handled." % peek(1), encoder
|
||||
end
|
||||
|
||||
else
|
||||
raise_inspect 'Unknown state: %p' % [state], encoder
|
||||
|
||||
end
|
||||
end
|
||||
|
||||
if [:string, :key].include? state
|
||||
encoder.end_group state
|
||||
end
|
||||
|
||||
encoder
|
||||
end
|
||||
|
||||
end
|
||||
|
||||
end
|
||||
end
|
||||
|
|
@ -0,0 +1,510 @@
|
|||
# encoding: ASCII-8BIT
|
||||
module CodeRay
|
||||
module Scanners
|
||||
|
||||
load :html
|
||||
|
||||
# Scanner for PHP.
|
||||
#
|
||||
# Original by Stefan Walk.
|
||||
class PHP < Scanner
|
||||
|
||||
register_for :php
|
||||
file_extension 'php'
|
||||
encoding 'BINARY'
|
||||
|
||||
KINDS_NOT_LOC = HTML::KINDS_NOT_LOC
|
||||
|
||||
protected
|
||||
|
||||
def setup
|
||||
@html_scanner = CodeRay.scanner :html, :tokens => @tokens, :keep_tokens => true, :keep_state => true
|
||||
end
|
||||
|
||||
def reset_instance
|
||||
super
|
||||
@html_scanner.reset
|
||||
end
|
||||
|
||||
module Words # :nodoc:
|
||||
|
||||
# according to http://www.php.net/manual/en/reserved.keywords.php
|
||||
KEYWORDS = %w[
|
||||
abstract and array as break case catch class clone const continue declare default do else elseif
|
||||
enddeclare endfor endforeach endif endswitch endwhile extends final for foreach function global
|
||||
goto if implements interface instanceof namespace new or private protected public static switch
|
||||
throw try use var while xor
|
||||
cfunction old_function
|
||||
]
|
||||
|
||||
TYPES = %w[ int integer float double bool boolean string array object resource ]
|
||||
|
||||
LANGUAGE_CONSTRUCTS = %w[
|
||||
die echo empty exit eval include include_once isset list
|
||||
require require_once return print unset
|
||||
]
|
||||
|
||||
CLASSES = %w[ Directory stdClass __PHP_Incomplete_Class exception php_user_filter Closure ]
|
||||
|
||||
# according to http://php.net/quickref.php on 2009-04-21;
|
||||
# all functions with _ excluded (module functions) and selected additional functions
|
||||
BUILTIN_FUNCTIONS = %w[
|
||||
abs acos acosh addcslashes addslashes aggregate array arsort ascii2ebcdic asin asinh asort assert atan atan2
|
||||
atanh basename bcadd bccomp bcdiv bcmod bcmul bcpow bcpowmod bcscale bcsqrt bcsub bin2hex bindec
|
||||
bindtextdomain bzclose bzcompress bzdecompress bzerrno bzerror bzerrstr bzflush bzopen bzread bzwrite
|
||||
calculhmac ceil chdir checkdate checkdnsrr chgrp chmod chop chown chr chroot clearstatcache closedir closelog
|
||||
compact constant copy cos cosh count crc32 crypt current date dcgettext dcngettext deaggregate decbin dechex
|
||||
decoct define defined deg2rad delete dgettext die dirname diskfreespace dl dngettext doubleval each
|
||||
ebcdic2ascii echo empty end ereg eregi escapeshellarg escapeshellcmd eval exec exit exp explode expm1 extract
|
||||
fclose feof fflush fgetc fgetcsv fgets fgetss file fileatime filectime filegroup fileinode filemtime fileowner
|
||||
fileperms filepro filesize filetype floatval flock floor flush fmod fnmatch fopen fpassthru fprintf fputcsv
|
||||
fputs fread frenchtojd fscanf fseek fsockopen fstat ftell ftok ftruncate fwrite getallheaders getcwd getdate
|
||||
getenv gethostbyaddr gethostbyname gethostbynamel getimagesize getlastmod getmxrr getmygid getmyinode getmypid
|
||||
getmyuid getopt getprotobyname getprotobynumber getrandmax getrusage getservbyname getservbyport gettext
|
||||
gettimeofday gettype glob gmdate gmmktime gmstrftime gregoriantojd gzclose gzcompress gzdecode gzdeflate
|
||||
gzencode gzeof gzfile gzgetc gzgets gzgetss gzinflate gzopen gzpassthru gzputs gzread gzrewind gzseek gztell
|
||||
gzuncompress gzwrite hash header hebrev hebrevc hexdec htmlentities htmlspecialchars hypot iconv idate
|
||||
implode include intval ip2long iptcembed iptcparse isset
|
||||
jddayofweek jdmonthname jdtofrench jdtogregorian jdtojewish jdtojulian jdtounix jewishtojd join jpeg2wbmp
|
||||
juliantojd key krsort ksort lcfirst lchgrp lchown levenshtein link linkinfo list localeconv localtime log
|
||||
log10 log1p long2ip lstat ltrim mail main max md5 metaphone mhash microtime min mkdir mktime msql natcasesort
|
||||
natsort next ngettext nl2br nthmac octdec opendir openlog
|
||||
ord overload pack passthru pathinfo pclose pfsockopen phpcredits phpinfo phpversion pi png2wbmp popen pos pow
|
||||
prev print printf putenv quotemeta rad2deg rand range rawurldecode rawurlencode readdir readfile readgzfile
|
||||
readline readlink realpath recode rename require reset rewind rewinddir rmdir round rsort rtrim scandir
|
||||
serialize setcookie setlocale setrawcookie settype sha1 shuffle signeurlpaiement sin sinh sizeof sleep snmpget
|
||||
snmpgetnext snmprealwalk snmpset snmpwalk snmpwalkoid sort soundex split spliti sprintf sqrt srand sscanf stat
|
||||
strcasecmp strchr strcmp strcoll strcspn strftime stripcslashes stripos stripslashes stristr strlen
|
||||
strnatcasecmp strnatcmp strncasecmp strncmp strpbrk strpos strptime strrchr strrev strripos strrpos strspn
|
||||
strstr strtok strtolower strtotime strtoupper strtr strval substr symlink syslog system tan tanh tempnam
|
||||
textdomain time tmpfile touch trim uasort ucfirst ucwords uksort umask uniqid unixtojd unlink unpack
|
||||
unserialize unset urldecode urlencode usleep usort vfprintf virtual vprintf vsprintf wordwrap
|
||||
array_change_key_case array_chunk array_combine array_count_values array_diff array_diff_assoc
|
||||
array_diff_key array_diff_uassoc array_diff_ukey array_fill array_fill_keys array_filter array_flip
|
||||
array_intersect array_intersect_assoc array_intersect_key array_intersect_uassoc array_intersect_ukey
|
||||
array_key_exists array_keys array_map array_merge array_merge_recursive array_multisort array_pad
|
||||
array_pop array_product array_push array_rand array_reduce array_reverse array_search array_shift
|
||||
array_slice array_splice array_sum array_udiff array_udiff_assoc array_udiff_uassoc array_uintersect
|
||||
array_uintersect_assoc array_uintersect_uassoc array_unique array_unshift array_values array_walk
|
||||
array_walk_recursive
|
||||
assert_options base_convert base64_decode base64_encode
|
||||
chunk_split class_exists class_implements class_parents
|
||||
count_chars debug_backtrace debug_print_backtrace debug_zval_dump
|
||||
error_get_last error_log error_reporting extension_loaded
|
||||
file_exists file_get_contents file_put_contents load_file
|
||||
func_get_arg func_get_args func_num_args function_exists
|
||||
get_browser get_called_class get_cfg_var get_class get_class_methods get_class_vars
|
||||
get_current_user get_declared_classes get_declared_interfaces get_defined_constants
|
||||
get_defined_functions get_defined_vars get_extension_funcs get_headers get_html_translation_table
|
||||
get_include_path get_included_files get_loaded_extensions get_magic_quotes_gpc get_magic_quotes_runtime
|
||||
get_meta_tags get_object_vars get_parent_class get_required_filesget_resource_type
|
||||
gc_collect_cycles gc_disable gc_enable gc_enabled
|
||||
halt_compiler headers_list headers_sent highlight_file highlight_string
|
||||
html_entity_decode htmlspecialchars_decode
|
||||
in_array include_once inclued_get_data
|
||||
is_a is_array is_binary is_bool is_buffer is_callable is_dir is_double is_executable is_file is_finite
|
||||
is_float is_infinite is_int is_integer is_link is_long is_nan is_null is_numeric is_object is_readable
|
||||
is_real is_resource is_scalar is_soap_fault is_string is_subclass_of is_unicode is_uploaded_file
|
||||
is_writable is_writeable
|
||||
locale_get_default locale_set_default
|
||||
number_format override_function parse_str parse_url
|
||||
php_check_syntax php_ini_loaded_file php_ini_scanned_files php_logo_guid php_sapi_name
|
||||
php_strip_whitespace php_uname
|
||||
preg_filter preg_grep preg_last_error preg_match preg_match_all preg_quote preg_replace
|
||||
preg_replace_callback preg_split print_r
|
||||
require_once register_shutdown_function register_tick_function
|
||||
set_error_handler set_exception_handler set_file_buffer set_include_path
|
||||
set_magic_quotes_runtime set_time_limit shell_exec
|
||||
str_getcsv str_ireplace str_pad str_repeat str_replace str_rot13 str_shuffle str_split str_word_count
|
||||
strip_tags substr_compare substr_count substr_replace
|
||||
time_nanosleep time_sleep_until
|
||||
token_get_all token_name trigger_error
|
||||
unregister_tick_function use_soap_error_handler user_error
|
||||
utf8_decode utf8_encode var_dump var_export
|
||||
version_compare
|
||||
zend_logo_guid zend_thread_id zend_version
|
||||
create_function call_user_func_array
|
||||
posix_access posix_ctermid posix_get_last_error posix_getcwd posix_getegid
|
||||
posix_geteuid posix_getgid posix_getgrgid posix_getgrnam posix_getgroups
|
||||
posix_getlogin posix_getpgid posix_getpgrp posix_getpid posix_getppid
|
||||
posix_getpwnam posix_getpwuid posix_getrlimit posix_getsid posix_getuid
|
||||
posix_initgroups posix_isatty posix_kill posix_mkfifo posix_mknod
|
||||
posix_setegid posix_seteuid posix_setgid posix_setpgid posix_setsid
|
||||
posix_setuid posix_strerror posix_times posix_ttyname posix_uname
|
||||
pcntl_alarm pcntl_exec pcntl_fork pcntl_getpriority pcntl_setpriority
|
||||
pcntl_signal pcntl_signal_dispatch pcntl_sigprocmask pcntl_sigtimedwait
|
||||
pcntl_sigwaitinfo pcntl_wait pcntl_waitpid pcntl_wexitstatus pcntl_wifexited
|
||||
pcntl_wifsignaled pcntl_wifstopped pcntl_wstopsig pcntl_wtermsig
|
||||
]
|
||||
# TODO: more built-in PHP functions?
|
||||
|
||||
EXCEPTIONS = %w[
|
||||
E_ERROR E_WARNING E_PARSE E_NOTICE E_CORE_ERROR E_CORE_WARNING E_COMPILE_ERROR E_COMPILE_WARNING
|
||||
E_USER_ERROR E_USER_WARNING E_USER_NOTICE E_DEPRECATED E_USER_DEPRECATED E_ALL E_STRICT
|
||||
]
|
||||
|
||||
CONSTANTS = %w[
|
||||
null true false self parent
|
||||
__LINE__ __DIR__ __FILE__ __LINE__
|
||||
__CLASS__ __NAMESPACE__ __METHOD__ __FUNCTION__
|
||||
PHP_VERSION PHP_MAJOR_VERSION PHP_MINOR_VERSION PHP_RELEASE_VERSION PHP_VERSION_ID PHP_EXTRA_VERSION PHP_ZTS
|
||||
PHP_DEBUG PHP_MAXPATHLEN PHP_OS PHP_SAPI PHP_EOL PHP_INT_MAX PHP_INT_SIZE DEFAULT_INCLUDE_PATH
|
||||
PEAR_INSTALL_DIR PEAR_EXTENSION_DIR PHP_EXTENSION_DIR PHP_PREFIX PHP_BINDIR PHP_LIBDIR PHP_DATADIR
|
||||
PHP_SYSCONFDIR PHP_LOCALSTATEDIR PHP_CONFIG_FILE_PATH PHP_CONFIG_FILE_SCAN_DIR PHP_SHLIB_SUFFIX
|
||||
PHP_OUTPUT_HANDLER_START PHP_OUTPUT_HANDLER_CONT PHP_OUTPUT_HANDLER_END
|
||||
__COMPILER_HALT_OFFSET__
|
||||
EXTR_OVERWRITE EXTR_SKIP EXTR_PREFIX_SAME EXTR_PREFIX_ALL EXTR_PREFIX_INVALID EXTR_PREFIX_IF_EXISTS
|
||||
EXTR_IF_EXISTS SORT_ASC SORT_DESC SORT_REGULAR SORT_NUMERIC SORT_STRING CASE_LOWER CASE_UPPER COUNT_NORMAL
|
||||
COUNT_RECURSIVE ASSERT_ACTIVE ASSERT_CALLBACK ASSERT_BAIL ASSERT_WARNING ASSERT_QUIET_EVAL CONNECTION_ABORTED
|
||||
CONNECTION_NORMAL CONNECTION_TIMEOUT INI_USER INI_PERDIR INI_SYSTEM INI_ALL M_E M_LOG2E M_LOG10E M_LN2 M_LN10
|
||||
M_PI M_PI_2 M_PI_4 M_1_PI M_2_PI M_2_SQRTPI M_SQRT2 M_SQRT1_2 CRYPT_SALT_LENGTH CRYPT_STD_DES CRYPT_EXT_DES
|
||||
CRYPT_MD5 CRYPT_BLOWFISH DIRECTORY_SEPARATOR SEEK_SET SEEK_CUR SEEK_END LOCK_SH LOCK_EX LOCK_UN LOCK_NB
|
||||
HTML_SPECIALCHARS HTML_ENTITIES ENT_COMPAT ENT_QUOTES ENT_NOQUOTES INFO_GENERAL INFO_CREDITS
|
||||
INFO_CONFIGURATION INFO_MODULES INFO_ENVIRONMENT INFO_VARIABLES INFO_LICENSE INFO_ALL CREDITS_GROUP
|
||||
CREDITS_GENERAL CREDITS_SAPI CREDITS_MODULES CREDITS_DOCS CREDITS_FULLPAGE CREDITS_QA CREDITS_ALL STR_PAD_LEFT
|
||||
STR_PAD_RIGHT STR_PAD_BOTH PATHINFO_DIRNAME PATHINFO_BASENAME PATHINFO_EXTENSION PATH_SEPARATOR CHAR_MAX
|
||||
LC_CTYPE LC_NUMERIC LC_TIME LC_COLLATE LC_MONETARY LC_ALL LC_MESSAGES ABDAY_1 ABDAY_2 ABDAY_3 ABDAY_4 ABDAY_5
|
||||
ABDAY_6 ABDAY_7 DAY_1 DAY_2 DAY_3 DAY_4 DAY_5 DAY_6 DAY_7 ABMON_1 ABMON_2 ABMON_3 ABMON_4 ABMON_5 ABMON_6
|
||||
ABMON_7 ABMON_8 ABMON_9 ABMON_10 ABMON_11 ABMON_12 MON_1 MON_2 MON_3 MON_4 MON_5 MON_6 MON_7 MON_8 MON_9
|
||||
MON_10 MON_11 MON_12 AM_STR PM_STR D_T_FMT D_FMT T_FMT T_FMT_AMPM ERA ERA_YEAR ERA_D_T_FMT ERA_D_FMT ERA_T_FMT
|
||||
ALT_DIGITS INT_CURR_SYMBOL CURRENCY_SYMBOL CRNCYSTR MON_DECIMAL_POINT MON_THOUSANDS_SEP MON_GROUPING
|
||||
POSITIVE_SIGN NEGATIVE_SIGN INT_FRAC_DIGITS FRAC_DIGITS P_CS_PRECEDES P_SEP_BY_SPACE N_CS_PRECEDES
|
||||
N_SEP_BY_SPACE P_SIGN_POSN N_SIGN_POSN DECIMAL_POINT RADIXCHAR THOUSANDS_SEP THOUSEP GROUPING YESEXPR NOEXPR
|
||||
YESSTR NOSTR CODESET LOG_EMERG LOG_ALERT LOG_CRIT LOG_ERR LOG_WARNING LOG_NOTICE LOG_INFO LOG_DEBUG LOG_KERN
|
||||
LOG_USER LOG_MAIL LOG_DAEMON LOG_AUTH LOG_SYSLOG LOG_LPR LOG_NEWS LOG_UUCP LOG_CRON LOG_AUTHPRIV LOG_LOCAL0
|
||||
LOG_LOCAL1 LOG_LOCAL2 LOG_LOCAL3 LOG_LOCAL4 LOG_LOCAL5 LOG_LOCAL6 LOG_LOCAL7 LOG_PID LOG_CONS LOG_ODELAY
|
||||
LOG_NDELAY LOG_NOWAIT LOG_PERROR
|
||||
]
|
||||
|
||||
PREDEFINED = %w[
|
||||
$GLOBALS $_SERVER $_GET $_POST $_FILES $_REQUEST $_SESSION $_ENV
|
||||
$_COOKIE $php_errormsg $HTTP_RAW_POST_DATA $http_response_header
|
||||
$argc $argv
|
||||
]
|
||||
|
||||
IDENT_KIND = WordList::CaseIgnoring.new(:ident).
|
||||
add(KEYWORDS, :keyword).
|
||||
add(TYPES, :predefined_type).
|
||||
add(LANGUAGE_CONSTRUCTS, :keyword).
|
||||
add(BUILTIN_FUNCTIONS, :predefined).
|
||||
add(CLASSES, :predefined_constant).
|
||||
add(EXCEPTIONS, :exception).
|
||||
add(CONSTANTS, :predefined_constant)
|
||||
|
||||
VARIABLE_KIND = WordList.new(:local_variable).
|
||||
add(PREDEFINED, :predefined)
|
||||
end
|
||||
|
||||
module RE # :nodoc:
|
||||
|
||||
PHP_START = /
|
||||
<script\s+[^>]*?language\s*=\s*"php"[^>]*?> |
|
||||
<script\s+[^>]*?language\s*=\s*'php'[^>]*?> |
|
||||
<\?php\d? |
|
||||
<\?(?!xml)
|
||||
/xi
|
||||
|
||||
PHP_END = %r!
|
||||
</script> |
|
||||
\?>
|
||||
!xi
|
||||
|
||||
HTML_INDICATOR = /<!DOCTYPE html|<(?:html|body|div|p)[> ]/i
|
||||
|
||||
IDENTIFIER = /[a-z_\x7f-\xFF][a-z0-9_\x7f-\xFF]*/i
|
||||
VARIABLE = /\$#{IDENTIFIER}/
|
||||
|
||||
OPERATOR = /
|
||||
\.(?!\d)=? | # dot that is not decimal point, string concatenation
|
||||
&& | \|\| | # logic
|
||||
:: | -> | => | # scope, member, dictionary
|
||||
\\(?!\n) | # namespace
|
||||
\+\+ | -- | # increment, decrement
|
||||
[,;?:()\[\]{}] | # simple delimiters
|
||||
[-+*\/%&|^]=? | # ordinary math, binary logic, assignment shortcuts
|
||||
[~$] | # whatever
|
||||
=& | # reference assignment
|
||||
[=!]=?=? | <> | # comparison and assignment
|
||||
<<=? | >>=? | [<>]=? # comparison and shift
|
||||
/x
|
||||
|
||||
end
|
||||
|
||||
protected
|
||||
|
||||
def scan_tokens encoder, options
|
||||
|
||||
if check(RE::PHP_START) || # starts with <?
|
||||
(match?(/\s*<\S/) && check(/.{1,1000}#{RE::PHP_START}/om)) || # starts with tag and contains <?
|
||||
check(/.{0,1000}#{RE::HTML_INDICATOR}/om) ||
|
||||
check(/.{1,100}#{RE::PHP_START}/om) # PHP start after max 100 chars
|
||||
# is HTML with embedded PHP, so start with HTML
|
||||
states = [:initial]
|
||||
else
|
||||
# is just PHP, so start with PHP surrounded by HTML
|
||||
states = [:initial, :php]
|
||||
end
|
||||
|
||||
label_expected = true
|
||||
case_expected = false
|
||||
|
||||
heredoc_delimiter = nil
|
||||
delimiter = nil
|
||||
modifier = nil
|
||||
|
||||
until eos?
|
||||
|
||||
case states.last
|
||||
|
||||
when :initial # HTML
|
||||
if match = scan(RE::PHP_START)
|
||||
encoder.text_token match, :inline_delimiter
|
||||
label_expected = true
|
||||
states << :php
|
||||
else
|
||||
match = scan_until(/(?=#{RE::PHP_START})/o) || scan_rest
|
||||
@html_scanner.tokenize match unless match.empty?
|
||||
end
|
||||
|
||||
when :php
|
||||
if match = scan(/\s+/)
|
||||
encoder.text_token match, :space
|
||||
|
||||
elsif match = scan(%r! (?m: \/\* (?: .*? \*\/ | .* ) ) | (?://|\#) .*? (?=#{RE::PHP_END}|$) !xo)
|
||||
encoder.text_token match, :comment
|
||||
|
||||
elsif match = scan(RE::IDENTIFIER)
|
||||
kind = Words::IDENT_KIND[match]
|
||||
if kind == :ident && label_expected && check(/:(?!:)/)
|
||||
kind = :label
|
||||
label_expected = true
|
||||
else
|
||||
label_expected = false
|
||||
if kind == :ident && match =~ /^[A-Z]/
|
||||
kind = :constant
|
||||
elsif kind == :keyword
|
||||
case match
|
||||
when 'class'
|
||||
states << :class_expected
|
||||
when 'function'
|
||||
states << :function_expected
|
||||
when 'case', 'default'
|
||||
case_expected = true
|
||||
end
|
||||
elsif match == 'b' && check(/['"]/) # binary string literal
|
||||
modifier = match
|
||||
next
|
||||
end
|
||||
end
|
||||
encoder.text_token match, kind
|
||||
|
||||
elsif match = scan(/(?:\d+\.\d*|\d*\.\d+)(?:e[-+]?\d+)?|\d+e[-+]?\d+/i)
|
||||
label_expected = false
|
||||
encoder.text_token match, :float
|
||||
|
||||
elsif match = scan(/0x[0-9a-fA-F]+/)
|
||||
label_expected = false
|
||||
encoder.text_token match, :hex
|
||||
|
||||
elsif match = scan(/\d+/)
|
||||
label_expected = false
|
||||
encoder.text_token match, :integer
|
||||
|
||||
elsif match = scan(/['"`]/)
|
||||
encoder.begin_group :string
|
||||
if modifier
|
||||
encoder.text_token modifier, :modifier
|
||||
modifier = nil
|
||||
end
|
||||
delimiter = match
|
||||
encoder.text_token match, :delimiter
|
||||
states.push match == "'" ? :sqstring : :dqstring
|
||||
|
||||
elsif match = scan(RE::VARIABLE)
|
||||
label_expected = false
|
||||
encoder.text_token match, Words::VARIABLE_KIND[match]
|
||||
|
||||
elsif match = scan(/\{/)
|
||||
encoder.text_token match, :operator
|
||||
label_expected = true
|
||||
states.push :php
|
||||
|
||||
elsif match = scan(/\}/)
|
||||
if states.size == 1
|
||||
encoder.text_token match, :error
|
||||
else
|
||||
states.pop
|
||||
if states.last.is_a?(::Array)
|
||||
delimiter = states.last[1]
|
||||
states[-1] = states.last[0]
|
||||
encoder.text_token match, :delimiter
|
||||
encoder.end_group :inline
|
||||
else
|
||||
encoder.text_token match, :operator
|
||||
label_expected = true
|
||||
end
|
||||
end
|
||||
|
||||
elsif match = scan(/@/)
|
||||
label_expected = false
|
||||
encoder.text_token match, :exception
|
||||
|
||||
elsif match = scan(RE::PHP_END)
|
||||
encoder.text_token match, :inline_delimiter
|
||||
states = [:initial]
|
||||
|
||||
elsif match = scan(/<<<(?:(#{RE::IDENTIFIER})|"(#{RE::IDENTIFIER})"|'(#{RE::IDENTIFIER})')/o)
|
||||
encoder.begin_group :string
|
||||
# warn 'heredoc in heredoc?' if heredoc_delimiter
|
||||
heredoc_delimiter = Regexp.escape(self[1] || self[2] || self[3])
|
||||
encoder.text_token match, :delimiter
|
||||
states.push self[3] ? :sqstring : :dqstring
|
||||
heredoc_delimiter = /#{heredoc_delimiter}(?=;?$)/
|
||||
|
||||
elsif match = scan(/#{RE::OPERATOR}/o)
|
||||
label_expected = match == ';'
|
||||
if case_expected
|
||||
label_expected = true if match == ':'
|
||||
case_expected = false
|
||||
end
|
||||
encoder.text_token match, :operator
|
||||
|
||||
else
|
||||
encoder.text_token getch, :error
|
||||
|
||||
end
|
||||
|
||||
when :sqstring
|
||||
if match = scan(heredoc_delimiter ? /[^\\\n]+/ : /[^'\\]+/)
|
||||
encoder.text_token match, :content
|
||||
elsif !heredoc_delimiter && match = scan(/'/)
|
||||
encoder.text_token match, :delimiter
|
||||
encoder.end_group :string
|
||||
delimiter = nil
|
||||
label_expected = false
|
||||
states.pop
|
||||
elsif heredoc_delimiter && match = scan(/\n/)
|
||||
if scan heredoc_delimiter
|
||||
encoder.text_token "\n", :content
|
||||
encoder.text_token matched, :delimiter
|
||||
encoder.end_group :string
|
||||
heredoc_delimiter = nil
|
||||
label_expected = false
|
||||
states.pop
|
||||
else
|
||||
encoder.text_token match, :content
|
||||
end
|
||||
elsif match = scan(heredoc_delimiter ? /\\\\/ : /\\[\\'\n]/)
|
||||
encoder.text_token match, :char
|
||||
elsif match = scan(/\\./m)
|
||||
encoder.text_token match, :content
|
||||
elsif match = scan(/\\/)
|
||||
encoder.text_token match, :error
|
||||
else
|
||||
states.pop
|
||||
end
|
||||
|
||||
when :dqstring
|
||||
if match = scan(heredoc_delimiter ? /[^${\\\n]+/ : (delimiter == '"' ? /[^"${\\]+/ : /[^`${\\]+/))
|
||||
encoder.text_token match, :content
|
||||
elsif !heredoc_delimiter && match = scan(delimiter == '"' ? /"/ : /`/)
|
||||
encoder.text_token match, :delimiter
|
||||
encoder.end_group :string
|
||||
delimiter = nil
|
||||
label_expected = false
|
||||
states.pop
|
||||
elsif heredoc_delimiter && match = scan(/\n/)
|
||||
if scan heredoc_delimiter
|
||||
encoder.text_token "\n", :content
|
||||
encoder.text_token matched, :delimiter
|
||||
encoder.end_group :string
|
||||
heredoc_delimiter = nil
|
||||
label_expected = false
|
||||
states.pop
|
||||
else
|
||||
encoder.text_token match, :content
|
||||
end
|
||||
elsif match = scan(/\\(?:x[0-9A-Fa-f]{1,2}|[0-7]{1,3})/)
|
||||
encoder.text_token match, :char
|
||||
elsif match = scan(heredoc_delimiter ? /\\[nrtvf\\$]/ : (delimiter == '"' ? /\\[nrtvf\\$"]/ : /\\[nrtvf\\$`]/))
|
||||
encoder.text_token match, :char
|
||||
elsif match = scan(/\\./m)
|
||||
encoder.text_token match, :content
|
||||
elsif match = scan(/\\/)
|
||||
encoder.text_token match, :error
|
||||
elsif match = scan(/#{RE::VARIABLE}/o)
|
||||
if check(/\[#{RE::IDENTIFIER}\]/o)
|
||||
encoder.begin_group :inline
|
||||
encoder.text_token match, :local_variable
|
||||
encoder.text_token scan(/\[/), :operator
|
||||
encoder.text_token scan(/#{RE::IDENTIFIER}/o), :ident
|
||||
encoder.text_token scan(/\]/), :operator
|
||||
encoder.end_group :inline
|
||||
elsif check(/\[/)
|
||||
match << scan(/\[['"]?#{RE::IDENTIFIER}?['"]?\]?/o)
|
||||
encoder.text_token match, :error
|
||||
elsif check(/->#{RE::IDENTIFIER}/o)
|
||||
encoder.begin_group :inline
|
||||
encoder.text_token match, :local_variable
|
||||
encoder.text_token scan(/->/), :operator
|
||||
encoder.text_token scan(/#{RE::IDENTIFIER}/o), :ident
|
||||
encoder.end_group :inline
|
||||
elsif check(/->/)
|
||||
match << scan(/->/)
|
||||
encoder.text_token match, :error
|
||||
else
|
||||
encoder.text_token match, :local_variable
|
||||
end
|
||||
elsif match = scan(/\{/)
|
||||
if check(/\$/)
|
||||
encoder.begin_group :inline
|
||||
states[-1] = [states.last, delimiter]
|
||||
delimiter = nil
|
||||
states.push :php
|
||||
encoder.text_token match, :delimiter
|
||||
else
|
||||
encoder.text_token match, :content
|
||||
end
|
||||
elsif match = scan(/\$\{#{RE::IDENTIFIER}\}/o)
|
||||
encoder.text_token match, :local_variable
|
||||
elsif match = scan(/\$/)
|
||||
encoder.text_token match, :content
|
||||
else
|
||||
states.pop
|
||||
end
|
||||
|
||||
when :class_expected
|
||||
if match = scan(/\s+/)
|
||||
encoder.text_token match, :space
|
||||
elsif match = scan(/#{RE::IDENTIFIER}/o)
|
||||
encoder.text_token match, :class
|
||||
states.pop
|
||||
else
|
||||
states.pop
|
||||
end
|
||||
|
||||
when :function_expected
|
||||
if match = scan(/\s+/)
|
||||
encoder.text_token match, :space
|
||||
elsif match = scan(/&/)
|
||||
encoder.text_token match, :operator
|
||||
elsif match = scan(/#{RE::IDENTIFIER}/o)
|
||||
encoder.text_token match, :function
|
||||
states.pop
|
||||
else
|
||||
states.pop
|
||||
end
|
||||
|
||||
else
|
||||
raise_inspect 'Unknown state!', encoder, states
|
||||
end
|
||||
|
||||
end
|
||||
|
||||
encoder
|
||||
end
|
||||
|
||||
end
|
||||
|
||||
end
|
||||
end
|
||||
|
|
@ -0,0 +1,287 @@
|
|||
module CodeRay
|
||||
module Scanners
|
||||
|
||||
# Scanner for Python. Supports Python 3.
|
||||
#
|
||||
# Based on pygments' PythonLexer, see
|
||||
# http://dev.pocoo.org/projects/pygments/browser/pygments/lexers/agile.py.
|
||||
class Python < Scanner
|
||||
|
||||
register_for :python
|
||||
file_extension 'py'
|
||||
|
||||
KEYWORDS = [
|
||||
'and', 'as', 'assert', 'break', 'class', 'continue', 'def',
|
||||
'del', 'elif', 'else', 'except', 'finally', 'for',
|
||||
'from', 'global', 'if', 'import', 'in', 'is', 'lambda', 'not',
|
||||
'or', 'pass', 'raise', 'return', 'try', 'while', 'with', 'yield',
|
||||
'nonlocal', # new in Python 3
|
||||
] # :nodoc:
|
||||
|
||||
OLD_KEYWORDS = [
|
||||
'exec', 'print', # gone in Python 3
|
||||
] # :nodoc:
|
||||
|
||||
PREDEFINED_METHODS_AND_TYPES = %w[
|
||||
__import__ abs all any apply basestring bin bool buffer
|
||||
bytearray bytes callable chr classmethod cmp coerce compile
|
||||
complex delattr dict dir divmod enumerate eval execfile exit
|
||||
file filter float frozenset getattr globals hasattr hash hex id
|
||||
input int intern isinstance issubclass iter len list locals
|
||||
long map max min next object oct open ord pow property range
|
||||
raw_input reduce reload repr reversed round set setattr slice
|
||||
sorted staticmethod str sum super tuple type unichr unicode
|
||||
vars xrange zip
|
||||
] # :nodoc:
|
||||
|
||||
PREDEFINED_EXCEPTIONS = %w[
|
||||
ArithmeticError AssertionError AttributeError
|
||||
BaseException DeprecationWarning EOFError EnvironmentError
|
||||
Exception FloatingPointError FutureWarning GeneratorExit IOError
|
||||
ImportError ImportWarning IndentationError IndexError KeyError
|
||||
KeyboardInterrupt LookupError MemoryError NameError
|
||||
NotImplemented NotImplementedError OSError OverflowError
|
||||
OverflowWarning PendingDeprecationWarning ReferenceError
|
||||
RuntimeError RuntimeWarning StandardError StopIteration
|
||||
SyntaxError SyntaxWarning SystemError SystemExit TabError
|
||||
TypeError UnboundLocalError UnicodeDecodeError
|
||||
UnicodeEncodeError UnicodeError UnicodeTranslateError
|
||||
UnicodeWarning UserWarning ValueError Warning ZeroDivisionError
|
||||
] # :nodoc:
|
||||
|
||||
PREDEFINED_VARIABLES_AND_CONSTANTS = [
|
||||
'False', 'True', 'None', # "keywords" since Python 3
|
||||
'self', 'Ellipsis', 'NotImplemented',
|
||||
] # :nodoc:
|
||||
|
||||
IDENT_KIND = WordList.new(:ident).
|
||||
add(KEYWORDS, :keyword).
|
||||
add(OLD_KEYWORDS, :old_keyword).
|
||||
add(PREDEFINED_METHODS_AND_TYPES, :predefined).
|
||||
add(PREDEFINED_VARIABLES_AND_CONSTANTS, :predefined_constant).
|
||||
add(PREDEFINED_EXCEPTIONS, :exception) # :nodoc:
|
||||
|
||||
NAME = / [[:alpha:]_] \w* /x # :nodoc:
|
||||
ESCAPE = / [abfnrtv\n\\'"] | x[a-fA-F0-9]{1,2} | [0-7]{1,3} /x # :nodoc:
|
||||
UNICODE_ESCAPE = / u[a-fA-F0-9]{4} | U[a-fA-F0-9]{8} | N\{[-\w ]+\} /x # :nodoc:
|
||||
|
||||
OPERATOR = /
|
||||
\.\.\. | # ellipsis
|
||||
\.(?!\d) | # dot but not decimal point
|
||||
[,;:()\[\]{}] | # simple delimiters
|
||||
\/\/=? | \*\*=? | # special math
|
||||
[-+*\/%&|^]=? | # ordinary math and binary logic
|
||||
[~`] | # binary complement and inspection
|
||||
<<=? | >>=? | [<>=]=? | != # comparison and assignment
|
||||
/x # :nodoc:
|
||||
|
||||
STRING_DELIMITER_REGEXP = Hash.new { |h, delimiter|
|
||||
h[delimiter] = Regexp.union delimiter # :nodoc:
|
||||
}
|
||||
|
||||
STRING_CONTENT_REGEXP = Hash.new { |h, delimiter|
|
||||
h[delimiter] = / [^\\\n]+? (?= \\ | $ | #{Regexp.escape(delimiter)} ) /x # :nodoc:
|
||||
}
|
||||
|
||||
DEF_NEW_STATE = WordList.new(:initial).
|
||||
add(%w(def), :def_expected).
|
||||
add(%w(import from), :include_expected).
|
||||
add(%w(class), :class_expected) # :nodoc:
|
||||
|
||||
DESCRIPTOR = /
|
||||
#{NAME}
|
||||
(?: \. #{NAME} )*
|
||||
| \*
|
||||
/x # :nodoc:
|
||||
|
||||
DOCSTRING_COMING = /
|
||||
[ \t]* u?r? ("""|''')
|
||||
/x # :nodoc:
|
||||
|
||||
protected
|
||||
|
||||
def scan_tokens encoder, options
|
||||
|
||||
state = :initial
|
||||
string_delimiter = nil
|
||||
string_raw = false
|
||||
string_type = nil
|
||||
docstring_coming = match?(/#{DOCSTRING_COMING}/o)
|
||||
last_token_dot = false
|
||||
unicode = string.respond_to?(:encoding) && string.encoding.name == 'UTF-8'
|
||||
from_import_state = []
|
||||
|
||||
until eos?
|
||||
|
||||
if state == :string
|
||||
if match = scan(STRING_DELIMITER_REGEXP[string_delimiter])
|
||||
encoder.text_token match, :delimiter
|
||||
encoder.end_group string_type
|
||||
string_type = nil
|
||||
state = :initial
|
||||
next
|
||||
elsif string_delimiter.size == 3 && match = scan(/\n/)
|
||||
encoder.text_token match, :content
|
||||
elsif match = scan(STRING_CONTENT_REGEXP[string_delimiter])
|
||||
encoder.text_token match, :content
|
||||
elsif !string_raw && match = scan(/ \\ #{ESCAPE} /ox)
|
||||
encoder.text_token match, :char
|
||||
elsif match = scan(/ \\ #{UNICODE_ESCAPE} /ox)
|
||||
encoder.text_token match, :char
|
||||
elsif match = scan(/ \\ . /x)
|
||||
encoder.text_token match, :content
|
||||
elsif match = scan(/ \\ | $ /x)
|
||||
encoder.end_group string_type
|
||||
string_type = nil
|
||||
encoder.text_token match, :error
|
||||
state = :initial
|
||||
else
|
||||
raise_inspect "else case \" reached; %p not handled." % peek(1), encoder, state
|
||||
end
|
||||
|
||||
elsif match = scan(/ [ \t]+ | \\?\n /x)
|
||||
encoder.text_token match, :space
|
||||
if match == "\n"
|
||||
state = :initial if state == :include_expected
|
||||
docstring_coming = true if match?(/#{DOCSTRING_COMING}/o)
|
||||
end
|
||||
next
|
||||
|
||||
elsif match = scan(/ \# [^\n]* /mx)
|
||||
encoder.text_token match, :comment
|
||||
next
|
||||
|
||||
elsif state == :initial
|
||||
|
||||
if match = scan(/#{OPERATOR}/o)
|
||||
encoder.text_token match, :operator
|
||||
|
||||
elsif match = scan(/(u?r?|b)?("""|"|'''|')/i)
|
||||
string_delimiter = self[2]
|
||||
string_type = docstring_coming ? :docstring : :string
|
||||
docstring_coming = false if docstring_coming
|
||||
encoder.begin_group string_type
|
||||
string_raw = false
|
||||
modifiers = self[1]
|
||||
unless modifiers.empty?
|
||||
string_raw = !!modifiers.index(?r)
|
||||
encoder.text_token modifiers, :modifier
|
||||
match = string_delimiter
|
||||
end
|
||||
state = :string
|
||||
encoder.text_token match, :delimiter
|
||||
|
||||
# TODO: backticks
|
||||
|
||||
elsif match = scan(unicode ? /#{NAME}/uo : /#{NAME}/o)
|
||||
kind = IDENT_KIND[match]
|
||||
# TODO: keyword arguments
|
||||
kind = :ident if last_token_dot
|
||||
if kind == :old_keyword
|
||||
kind = check(/\(/) ? :ident : :keyword
|
||||
elsif kind == :predefined && check(/ *=/)
|
||||
kind = :ident
|
||||
elsif kind == :keyword
|
||||
state = DEF_NEW_STATE[match]
|
||||
from_import_state << match.to_sym if state == :include_expected
|
||||
end
|
||||
encoder.text_token match, kind
|
||||
|
||||
elsif match = scan(/@[a-zA-Z0-9_.]+[lL]?/)
|
||||
encoder.text_token match, :decorator
|
||||
|
||||
elsif match = scan(/0[xX][0-9A-Fa-f]+[lL]?/)
|
||||
encoder.text_token match, :hex
|
||||
|
||||
elsif match = scan(/0[bB][01]+[lL]?/)
|
||||
encoder.text_token match, :binary
|
||||
|
||||
elsif match = scan(/(?:\d*\.\d+|\d+\.\d*)(?:[eE][+-]?\d+)?|\d+[eE][+-]?\d+/)
|
||||
if scan(/[jJ]/)
|
||||
match << matched
|
||||
encoder.text_token match, :imaginary
|
||||
else
|
||||
encoder.text_token match, :float
|
||||
end
|
||||
|
||||
elsif match = scan(/0[oO][0-7]+|0[0-7]+(?![89.eE])[lL]?/)
|
||||
encoder.text_token match, :octal
|
||||
|
||||
elsif match = scan(/\d+([lL])?/)
|
||||
if self[1] == nil && scan(/[jJ]/)
|
||||
match << matched
|
||||
encoder.text_token match, :imaginary
|
||||
else
|
||||
encoder.text_token match, :integer
|
||||
end
|
||||
|
||||
else
|
||||
encoder.text_token getch, :error
|
||||
|
||||
end
|
||||
|
||||
elsif state == :def_expected
|
||||
state = :initial
|
||||
if match = scan(unicode ? /#{NAME}/uo : /#{NAME}/o)
|
||||
encoder.text_token match, :method
|
||||
else
|
||||
next
|
||||
end
|
||||
|
||||
elsif state == :class_expected
|
||||
state = :initial
|
||||
if match = scan(unicode ? /#{NAME}/uo : /#{NAME}/o)
|
||||
encoder.text_token match, :class
|
||||
else
|
||||
next
|
||||
end
|
||||
|
||||
elsif state == :include_expected
|
||||
if match = scan(unicode ? /#{DESCRIPTOR}/uo : /#{DESCRIPTOR}/o)
|
||||
if match == 'as'
|
||||
encoder.text_token match, :keyword
|
||||
from_import_state << :as
|
||||
elsif from_import_state.first == :from && match == 'import'
|
||||
encoder.text_token match, :keyword
|
||||
from_import_state << :import
|
||||
elsif from_import_state.last == :as
|
||||
# encoder.text_token match, match[0,1][unicode ? /[[:upper:]]/u : /[[:upper:]]/] ? :class : :method
|
||||
encoder.text_token match, :ident
|
||||
from_import_state.pop
|
||||
elsif IDENT_KIND[match] == :keyword
|
||||
unscan
|
||||
match = nil
|
||||
state = :initial
|
||||
next
|
||||
else
|
||||
encoder.text_token match, :include
|
||||
end
|
||||
elsif match = scan(/,/)
|
||||
from_import_state.pop if from_import_state.last == :as
|
||||
encoder.text_token match, :operator
|
||||
else
|
||||
from_import_state = []
|
||||
state = :initial
|
||||
next
|
||||
end
|
||||
|
||||
else
|
||||
raise_inspect 'Unknown state', encoder, state
|
||||
|
||||
end
|
||||
|
||||
last_token_dot = match == '.'
|
||||
|
||||
end
|
||||
|
||||
if state == :string
|
||||
encoder.end_group string_type
|
||||
end
|
||||
|
||||
encoder
|
||||
end
|
||||
|
||||
end
|
||||
|
||||
end
|
||||
end
|
||||
|
|
@ -0,0 +1,66 @@
|
|||
module CodeRay
|
||||
module Scanners
|
||||
|
||||
# = Debug Scanner
|
||||
#
|
||||
# Parses the output of the Encoders::Debug encoder.
|
||||
class Raydebug < Scanner
|
||||
|
||||
register_for :raydebug
|
||||
file_extension 'raydebug'
|
||||
title 'CodeRay Token Dump'
|
||||
|
||||
protected
|
||||
|
||||
def scan_tokens encoder, options
|
||||
|
||||
opened_tokens = []
|
||||
|
||||
until eos?
|
||||
|
||||
if match = scan(/\s+/)
|
||||
encoder.text_token match, :space
|
||||
|
||||
elsif match = scan(/ (\w+) \( ( [^\)\\]* ( \\. [^\)\\]* )* ) /x)
|
||||
kind = self[1]
|
||||
encoder.text_token kind, :class
|
||||
encoder.text_token '(', :operator
|
||||
match = self[2]
|
||||
encoder.text_token match, kind.to_sym
|
||||
encoder.text_token match, :operator if match = scan(/\)/)
|
||||
|
||||
elsif match = scan(/ (\w+) ([<\[]) /x)
|
||||
kind = self[1]
|
||||
case self[2]
|
||||
when '<'
|
||||
encoder.text_token kind, :class
|
||||
when '['
|
||||
encoder.text_token kind, :class
|
||||
else
|
||||
raise 'CodeRay bug: This case should not be reached.'
|
||||
end
|
||||
kind = kind.to_sym
|
||||
opened_tokens << kind
|
||||
encoder.begin_group kind
|
||||
encoder.text_token self[2], :operator
|
||||
|
||||
elsif !opened_tokens.empty? && match = scan(/ [>\]] /x)
|
||||
encoder.text_token match, :operator
|
||||
encoder.end_group opened_tokens.pop
|
||||
|
||||
else
|
||||
encoder.text_token getch, :space
|
||||
|
||||
end
|
||||
|
||||
end
|
||||
|
||||
encoder.end_group opened_tokens.pop until opened_tokens.empty?
|
||||
|
||||
encoder
|
||||
end
|
||||
|
||||
end
|
||||
|
||||
end
|
||||
end
|
||||
|
|
@ -0,0 +1,461 @@
|
|||
module CodeRay
|
||||
module Scanners
|
||||
|
||||
# This scanner is really complex, since Ruby _is_ a complex language!
|
||||
#
|
||||
# It tries to highlight 100% of all common code,
|
||||
# and 90% of strange codes.
|
||||
#
|
||||
# It is optimized for HTML highlighting, and is not very useful for
|
||||
# parsing or pretty printing.
|
||||
class Ruby < Scanner
|
||||
|
||||
register_for :ruby
|
||||
file_extension 'rb'
|
||||
|
||||
autoload :Patterns, CodeRay.coderay_path('scanners', 'ruby', 'patterns')
|
||||
autoload :StringState, CodeRay.coderay_path('scanners', 'ruby', 'string_state')
|
||||
|
||||
def interpreted_string_state
|
||||
StringState.new :string, true, '"'
|
||||
end
|
||||
|
||||
protected
|
||||
|
||||
def setup
|
||||
@state = :initial
|
||||
end
|
||||
|
||||
def scan_tokens encoder, options
|
||||
state, heredocs = options[:state] || @state
|
||||
heredocs = heredocs.dup if heredocs.is_a?(Array)
|
||||
|
||||
if state && state.instance_of?(StringState)
|
||||
encoder.begin_group state.type
|
||||
end
|
||||
|
||||
last_state = nil
|
||||
|
||||
method_call_expected = false
|
||||
value_expected = true
|
||||
|
||||
inline_block_stack = nil
|
||||
inline_block_curly_depth = 0
|
||||
|
||||
if heredocs
|
||||
state = heredocs.shift
|
||||
encoder.begin_group state.type
|
||||
heredocs = nil if heredocs.empty?
|
||||
end
|
||||
|
||||
# def_object_stack = nil
|
||||
# def_object_paren_depth = 0
|
||||
|
||||
patterns = Patterns # avoid constant lookup
|
||||
|
||||
unicode = string.respond_to?(:encoding) && string.encoding.name == 'UTF-8'
|
||||
|
||||
until eos?
|
||||
|
||||
if state.instance_of? ::Symbol
|
||||
|
||||
if match = scan(/[ \t\f\v]+/)
|
||||
encoder.text_token match, :space
|
||||
|
||||
elsif match = scan(/\n/)
|
||||
if heredocs
|
||||
unscan # heredoc scanning needs \n at start
|
||||
state = heredocs.shift
|
||||
encoder.begin_group state.type
|
||||
heredocs = nil if heredocs.empty?
|
||||
else
|
||||
state = :initial if state == :undef_comma_expected
|
||||
encoder.text_token match, :space
|
||||
value_expected = true
|
||||
end
|
||||
|
||||
elsif match = scan(bol? ? / \#(!)?.* | #{patterns::RUBYDOC_OR_DATA} /ox : /\#.*/)
|
||||
encoder.text_token match, self[1] ? :doctype : :comment
|
||||
|
||||
elsif match = scan(/\\\n/)
|
||||
if heredocs
|
||||
unscan # heredoc scanning needs \n at start
|
||||
encoder.text_token scan(/\\/), :space
|
||||
state = heredocs.shift
|
||||
encoder.begin_group state.type
|
||||
heredocs = nil if heredocs.empty?
|
||||
else
|
||||
encoder.text_token match, :space
|
||||
end
|
||||
|
||||
elsif state == :initial
|
||||
|
||||
# IDENTS #
|
||||
if !method_call_expected &&
|
||||
match = scan(unicode ? /#{patterns::METHOD_NAME}/uo :
|
||||
/#{patterns::METHOD_NAME}/o)
|
||||
value_expected = false
|
||||
kind = patterns::IDENT_KIND[match]
|
||||
if kind == :ident
|
||||
if match[/\A[A-Z]/] && !(match[/[!?]$/] || match?(/\(/))
|
||||
kind = :constant
|
||||
end
|
||||
elsif kind == :keyword
|
||||
state = patterns::KEYWORD_NEW_STATE[match]
|
||||
value_expected = true if patterns::KEYWORDS_EXPECTING_VALUE[match]
|
||||
end
|
||||
value_expected = true if !value_expected && check(/#{patterns::VALUE_FOLLOWS}/o)
|
||||
encoder.text_token match, kind
|
||||
|
||||
elsif method_call_expected &&
|
||||
match = scan(unicode ? /#{patterns::METHOD_AFTER_DOT}/uo :
|
||||
/#{patterns::METHOD_AFTER_DOT}/o)
|
||||
if method_call_expected == '::' && match[/\A[A-Z]/] && !match?(/\(/)
|
||||
encoder.text_token match, :constant
|
||||
else
|
||||
encoder.text_token match, :ident
|
||||
end
|
||||
method_call_expected = false
|
||||
value_expected = check(/#{patterns::VALUE_FOLLOWS}/o)
|
||||
|
||||
# OPERATORS #
|
||||
elsif !method_call_expected && match = scan(/ (\.(?!\.)|::) | (?: \.\.\.? | ==?=? | [,\(\[\{] )() | [\)\]\}] /x)
|
||||
method_call_expected = self[1]
|
||||
value_expected = !method_call_expected && self[2]
|
||||
if inline_block_stack
|
||||
case match
|
||||
when '{'
|
||||
inline_block_curly_depth += 1
|
||||
when '}'
|
||||
inline_block_curly_depth -= 1
|
||||
if inline_block_curly_depth == 0 # closing brace of inline block reached
|
||||
state, inline_block_curly_depth, heredocs = inline_block_stack.pop
|
||||
inline_block_stack = nil if inline_block_stack.empty?
|
||||
heredocs = nil if heredocs && heredocs.empty?
|
||||
encoder.text_token match, :inline_delimiter
|
||||
encoder.end_group :inline
|
||||
next
|
||||
end
|
||||
end
|
||||
end
|
||||
encoder.text_token match, :operator
|
||||
|
||||
elsif match = scan(unicode ? /#{patterns::SYMBOL}/uo :
|
||||
/#{patterns::SYMBOL}/o)
|
||||
case delim = match[1]
|
||||
when ?', ?"
|
||||
encoder.begin_group :symbol
|
||||
encoder.text_token ':', :symbol
|
||||
match = delim.chr
|
||||
encoder.text_token match, :delimiter
|
||||
state = self.class::StringState.new :symbol, delim == ?", match
|
||||
else
|
||||
encoder.text_token match, :symbol
|
||||
value_expected = false
|
||||
end
|
||||
|
||||
elsif match = scan(/ ' (?:(?>[^'\\]*) ')? | " (?:(?>[^"\\\#]*) ")? /mx)
|
||||
encoder.begin_group :string
|
||||
if match.size == 1
|
||||
encoder.text_token match, :delimiter
|
||||
state = self.class::StringState.new :string, match == '"', match # important for streaming
|
||||
else
|
||||
encoder.text_token match[0,1], :delimiter
|
||||
encoder.text_token match[1..-2], :content if match.size > 2
|
||||
encoder.text_token match[-1,1], :delimiter
|
||||
encoder.end_group :string
|
||||
value_expected = false
|
||||
end
|
||||
|
||||
elsif match = scan(unicode ? /#{patterns::INSTANCE_VARIABLE}/uo :
|
||||
/#{patterns::INSTANCE_VARIABLE}/o)
|
||||
value_expected = false
|
||||
encoder.text_token match, :instance_variable
|
||||
|
||||
elsif value_expected && match = scan(/\//)
|
||||
encoder.begin_group :regexp
|
||||
encoder.text_token match, :delimiter
|
||||
state = self.class::StringState.new :regexp, true, '/'
|
||||
|
||||
elsif match = scan(value_expected ? /[-+]?#{patterns::NUMERIC}/o : /#{patterns::NUMERIC}/o)
|
||||
if method_call_expected
|
||||
encoder.text_token match, :error
|
||||
method_call_expected = false
|
||||
else
|
||||
encoder.text_token match, self[1] ? :float : :integer # TODO: send :hex/:octal/:binary
|
||||
end
|
||||
value_expected = false
|
||||
|
||||
elsif match = scan(/ [-+!~^\/]=? | [:;] | [*|&]{1,2}=? | >>? /x)
|
||||
value_expected = true
|
||||
encoder.text_token match, :operator
|
||||
|
||||
elsif value_expected && match = scan(/#{patterns::HEREDOC_OPEN}/o)
|
||||
quote = self[3]
|
||||
delim = self[quote ? 4 : 2]
|
||||
kind = patterns::QUOTE_TO_TYPE[quote]
|
||||
encoder.begin_group kind
|
||||
encoder.text_token match, :delimiter
|
||||
encoder.end_group kind
|
||||
heredocs ||= [] # create heredocs if empty
|
||||
heredocs << self.class::StringState.new(kind, quote != "'", delim,
|
||||
self[1] == '-' ? :indented : :linestart)
|
||||
value_expected = false
|
||||
|
||||
elsif value_expected && match = scan(/#{patterns::FANCY_STRING_START}/o)
|
||||
kind = patterns::FANCY_STRING_KIND[self[1]]
|
||||
encoder.begin_group kind
|
||||
state = self.class::StringState.new kind, patterns::FANCY_STRING_INTERPRETED[self[1]], self[2]
|
||||
encoder.text_token match, :delimiter
|
||||
|
||||
elsif value_expected && match = scan(/#{patterns::CHARACTER}/o)
|
||||
value_expected = false
|
||||
encoder.text_token match, :integer
|
||||
|
||||
elsif match = scan(/ %=? | <(?:<|=>?)? | \? /x)
|
||||
value_expected = true
|
||||
encoder.text_token match, :operator
|
||||
|
||||
elsif match = scan(/`/)
|
||||
encoder.begin_group :shell
|
||||
encoder.text_token match, :delimiter
|
||||
state = self.class::StringState.new :shell, true, match
|
||||
|
||||
elsif match = scan(unicode ? /#{patterns::GLOBAL_VARIABLE}/uo :
|
||||
/#{patterns::GLOBAL_VARIABLE}/o)
|
||||
encoder.text_token match, :global_variable
|
||||
value_expected = false
|
||||
|
||||
elsif match = scan(unicode ? /#{patterns::CLASS_VARIABLE}/uo :
|
||||
/#{patterns::CLASS_VARIABLE}/o)
|
||||
encoder.text_token match, :class_variable
|
||||
value_expected = false
|
||||
|
||||
elsif match = scan(/\\\z/)
|
||||
encoder.text_token match, :space
|
||||
|
||||
else
|
||||
if method_call_expected
|
||||
method_call_expected = false
|
||||
next
|
||||
end
|
||||
unless unicode
|
||||
# check for unicode
|
||||
$DEBUG_BEFORE, $DEBUG = $DEBUG, false
|
||||
begin
|
||||
if check(/./mu).size > 1
|
||||
# seems like we should try again with unicode
|
||||
unicode = true
|
||||
end
|
||||
rescue
|
||||
# bad unicode char; use getch
|
||||
ensure
|
||||
$DEBUG = $DEBUG_BEFORE
|
||||
end
|
||||
next if unicode
|
||||
end
|
||||
|
||||
encoder.text_token getch, :error
|
||||
|
||||
end
|
||||
|
||||
if last_state
|
||||
state = last_state
|
||||
last_state = nil
|
||||
end
|
||||
|
||||
elsif state == :def_expected
|
||||
if match = scan(unicode ? /(?>#{patterns::METHOD_NAME_EX})(?!\.|::)/uo :
|
||||
/(?>#{patterns::METHOD_NAME_EX})(?!\.|::)/o)
|
||||
encoder.text_token match, :method
|
||||
state = :initial
|
||||
else
|
||||
last_state = :dot_expected
|
||||
state = :initial
|
||||
end
|
||||
|
||||
elsif state == :dot_expected
|
||||
if match = scan(/\.|::/)
|
||||
# invalid definition
|
||||
state = :def_expected
|
||||
encoder.text_token match, :operator
|
||||
else
|
||||
state = :initial
|
||||
end
|
||||
|
||||
elsif state == :module_expected
|
||||
if match = scan(/<</)
|
||||
encoder.text_token match, :operator
|
||||
else
|
||||
state = :initial
|
||||
if match = scan(unicode ? / (?:#{patterns::IDENT}::)* #{patterns::IDENT} /oux :
|
||||
/ (?:#{patterns::IDENT}::)* #{patterns::IDENT} /ox)
|
||||
encoder.text_token match, :class
|
||||
end
|
||||
end
|
||||
|
||||
elsif state == :undef_expected
|
||||
state = :undef_comma_expected
|
||||
if match = scan(unicode ? /(?>#{patterns::METHOD_NAME_EX})(?!\.|::)/uo :
|
||||
/(?>#{patterns::METHOD_NAME_EX})(?!\.|::)/o)
|
||||
encoder.text_token match, :method
|
||||
elsif match = scan(/#{patterns::SYMBOL}/o)
|
||||
case delim = match[1]
|
||||
when ?', ?"
|
||||
encoder.begin_group :symbol
|
||||
encoder.text_token ':', :symbol
|
||||
match = delim.chr
|
||||
encoder.text_token match, :delimiter
|
||||
state = self.class::StringState.new :symbol, delim == ?", match
|
||||
state.next_state = :undef_comma_expected
|
||||
else
|
||||
encoder.text_token match, :symbol
|
||||
end
|
||||
else
|
||||
state = :initial
|
||||
end
|
||||
|
||||
elsif state == :undef_comma_expected
|
||||
if match = scan(/,/)
|
||||
encoder.text_token match, :operator
|
||||
state = :undef_expected
|
||||
else
|
||||
state = :initial
|
||||
end
|
||||
|
||||
elsif state == :alias_expected
|
||||
match = scan(unicode ? /(#{patterns::METHOD_NAME_OR_SYMBOL})([ \t]+)(#{patterns::METHOD_NAME_OR_SYMBOL})/uo :
|
||||
/(#{patterns::METHOD_NAME_OR_SYMBOL})([ \t]+)(#{patterns::METHOD_NAME_OR_SYMBOL})/o)
|
||||
|
||||
if match
|
||||
encoder.text_token self[1], (self[1][0] == ?: ? :symbol : :method)
|
||||
encoder.text_token self[2], :space
|
||||
encoder.text_token self[3], (self[3][0] == ?: ? :symbol : :method)
|
||||
end
|
||||
state = :initial
|
||||
|
||||
else
|
||||
#:nocov:
|
||||
raise_inspect 'Unknown state: %p' % [state], encoder
|
||||
#:nocov:
|
||||
end
|
||||
|
||||
else # StringState
|
||||
|
||||
match = scan_until(state.pattern) || scan_rest
|
||||
unless match.empty?
|
||||
encoder.text_token match, :content
|
||||
break if eos?
|
||||
end
|
||||
|
||||
if state.heredoc && self[1] # end of heredoc
|
||||
match = getch
|
||||
match << scan_until(/$/) unless eos?
|
||||
encoder.text_token match, :delimiter unless match.empty?
|
||||
encoder.end_group state.type
|
||||
state = state.next_state
|
||||
next
|
||||
end
|
||||
|
||||
case match = getch
|
||||
|
||||
when state.delim
|
||||
if state.paren_depth
|
||||
state.paren_depth -= 1
|
||||
if state.paren_depth > 0
|
||||
encoder.text_token match, :content
|
||||
next
|
||||
end
|
||||
end
|
||||
encoder.text_token match, :delimiter
|
||||
if state.type == :regexp && !eos?
|
||||
match = scan(/#{patterns::REGEXP_MODIFIERS}/o)
|
||||
encoder.text_token match, :modifier unless match.empty?
|
||||
end
|
||||
encoder.end_group state.type
|
||||
value_expected = false
|
||||
state = state.next_state
|
||||
|
||||
when '\\'
|
||||
if state.interpreted
|
||||
if esc = scan(/#{patterns::ESCAPE}/o)
|
||||
encoder.text_token match + esc, :char
|
||||
else
|
||||
encoder.text_token match, :error
|
||||
end
|
||||
else
|
||||
case esc = getch
|
||||
when nil
|
||||
encoder.text_token match, :content
|
||||
when state.delim, '\\'
|
||||
encoder.text_token match + esc, :char
|
||||
else
|
||||
encoder.text_token match + esc, :content
|
||||
end
|
||||
end
|
||||
|
||||
when '#'
|
||||
case peek(1)
|
||||
when '{'
|
||||
inline_block_stack ||= []
|
||||
inline_block_stack << [state, inline_block_curly_depth, heredocs]
|
||||
value_expected = true
|
||||
state = :initial
|
||||
inline_block_curly_depth = 1
|
||||
encoder.begin_group :inline
|
||||
encoder.text_token match + getch, :inline_delimiter
|
||||
when '$', '@'
|
||||
encoder.text_token match, :escape
|
||||
last_state = state
|
||||
state = :initial
|
||||
else
|
||||
#:nocov:
|
||||
raise_inspect 'else-case # reached; #%p not handled' % [peek(1)], encoder
|
||||
#:nocov:
|
||||
end
|
||||
|
||||
when state.opening_paren
|
||||
state.paren_depth += 1
|
||||
encoder.text_token match, :content
|
||||
|
||||
else
|
||||
#:nocov
|
||||
raise_inspect 'else-case " reached; %p not handled, state = %p' % [match, state], encoder
|
||||
#:nocov:
|
||||
|
||||
end
|
||||
|
||||
end
|
||||
|
||||
end
|
||||
|
||||
# cleaning up
|
||||
if state.is_a? StringState
|
||||
encoder.end_group state.type
|
||||
end
|
||||
|
||||
if options[:keep_state]
|
||||
if state.is_a?(StringState) && state.heredoc
|
||||
(heredocs ||= []).unshift state
|
||||
state = :initial
|
||||
elsif heredocs && heredocs.empty?
|
||||
heredocs = nil
|
||||
end
|
||||
@state = state, heredocs
|
||||
end
|
||||
|
||||
if inline_block_stack
|
||||
until inline_block_stack.empty?
|
||||
state, = *inline_block_stack.pop
|
||||
encoder.end_group :inline
|
||||
encoder.end_group state.type
|
||||
end
|
||||
end
|
||||
|
||||
encoder
|
||||
end
|
||||
|
||||
end
|
||||
|
||||
end
|
||||
end
|
||||
|
|
@ -0,0 +1,175 @@
|
|||
# encoding: utf-8
|
||||
module CodeRay
|
||||
module Scanners
|
||||
|
||||
module Ruby::Patterns # :nodoc: all
|
||||
|
||||
KEYWORDS = %w[
|
||||
and def end in or unless begin
|
||||
defined? ensure module redo super until
|
||||
BEGIN break do next rescue then
|
||||
when END case else for retry
|
||||
while alias class elsif if not return
|
||||
undef yield
|
||||
]
|
||||
|
||||
# See http://murfy.de/ruby-constants.
|
||||
PREDEFINED_CONSTANTS = %w[
|
||||
nil true false self
|
||||
DATA ARGV ARGF ENV
|
||||
FALSE TRUE NIL
|
||||
STDERR STDIN STDOUT
|
||||
TOPLEVEL_BINDING
|
||||
RUBY_COPYRIGHT RUBY_DESCRIPTION RUBY_ENGINE RUBY_PATCHLEVEL
|
||||
RUBY_PLATFORM RUBY_RELEASE_DATE RUBY_REVISION RUBY_VERSION
|
||||
__FILE__ __LINE__ __ENCODING__
|
||||
]
|
||||
|
||||
IDENT_KIND = WordList.new(:ident).
|
||||
add(KEYWORDS, :keyword).
|
||||
add(PREDEFINED_CONSTANTS, :predefined_constant)
|
||||
|
||||
KEYWORD_NEW_STATE = WordList.new(:initial).
|
||||
add(%w[ def ], :def_expected).
|
||||
add(%w[ undef ], :undef_expected).
|
||||
add(%w[ alias ], :alias_expected).
|
||||
add(%w[ class module ], :module_expected)
|
||||
|
||||
IDENT = 'ä'[/[[:alpha:]]/] == 'ä' ? /[[:alpha:]_][[:alnum:]_]*/ : /[^\W\d]\w*/
|
||||
|
||||
METHOD_NAME = / #{IDENT} [?!]? /ox
|
||||
METHOD_NAME_OPERATOR = /
|
||||
\*\*? # multiplication and power
|
||||
| [-+~]@? # plus, minus, tilde with and without at sign
|
||||
| [\/%&|^`] # division, modulo or format strings, and, or, xor, system
|
||||
| \[\]=? # array getter and setter
|
||||
| << | >> # append or shift left, shift right
|
||||
| <=?>? | >=? # comparison, rocket operator
|
||||
| ===? | =~ # simple equality, case equality, match
|
||||
| ![~=@]? # negation with and without at sign, not-equal and not-match
|
||||
/ox
|
||||
METHOD_SUFFIX = / (?: [?!] | = (?![~>]|=(?!>)) ) /x
|
||||
METHOD_NAME_EX = / #{IDENT} #{METHOD_SUFFIX}? | #{METHOD_NAME_OPERATOR} /ox
|
||||
METHOD_AFTER_DOT = / #{IDENT} [?!]? | #{METHOD_NAME_OPERATOR} /ox
|
||||
INSTANCE_VARIABLE = / @ #{IDENT} /ox
|
||||
CLASS_VARIABLE = / @@ #{IDENT} /ox
|
||||
OBJECT_VARIABLE = / @@? #{IDENT} /ox
|
||||
GLOBAL_VARIABLE = / \$ (?: #{IDENT} | [1-9]\d* | 0\w* | [~&+`'=\/,;_.<>!@$?*":\\] | -[a-zA-Z_0-9] ) /ox
|
||||
PREFIX_VARIABLE = / #{GLOBAL_VARIABLE} | #{OBJECT_VARIABLE} /ox
|
||||
VARIABLE = / @?@? #{IDENT} | #{GLOBAL_VARIABLE} /ox
|
||||
|
||||
QUOTE_TO_TYPE = {
|
||||
'`' => :shell,
|
||||
'/'=> :regexp,
|
||||
}
|
||||
QUOTE_TO_TYPE.default = :string
|
||||
|
||||
REGEXP_MODIFIERS = /[mousenix]*/
|
||||
|
||||
DECIMAL = /\d+(?:_\d+)*/
|
||||
OCTAL = /0_?[0-7]+(?:_[0-7]+)*/
|
||||
HEXADECIMAL = /0x[0-9A-Fa-f]+(?:_[0-9A-Fa-f]+)*/
|
||||
BINARY = /0b[01]+(?:_[01]+)*/
|
||||
|
||||
EXPONENT = / [eE] [+-]? #{DECIMAL} /ox
|
||||
FLOAT_SUFFIX = / #{EXPONENT} | \. #{DECIMAL} #{EXPONENT}? /ox
|
||||
FLOAT_OR_INT = / #{DECIMAL} (?: #{FLOAT_SUFFIX} () )? /ox
|
||||
NUMERIC = / (?: (?=0) (?: #{OCTAL} | #{HEXADECIMAL} | #{BINARY} ) | #{FLOAT_OR_INT} ) /ox
|
||||
|
||||
SYMBOL = /
|
||||
:
|
||||
(?:
|
||||
#{METHOD_NAME_EX}
|
||||
| #{PREFIX_VARIABLE}
|
||||
| ['"]
|
||||
)
|
||||
/ox
|
||||
METHOD_NAME_OR_SYMBOL = / #{METHOD_NAME_EX} | #{SYMBOL} /ox
|
||||
|
||||
SIMPLE_ESCAPE = /
|
||||
[abefnrstv]
|
||||
| [0-7]{1,3}
|
||||
| x[0-9A-Fa-f]{1,2}
|
||||
| .
|
||||
/mx
|
||||
|
||||
CONTROL_META_ESCAPE = /
|
||||
(?: M-|C-|c )
|
||||
(?: \\ (?: M-|C-|c ) )*
|
||||
(?: [^\\] | \\ #{SIMPLE_ESCAPE} )?
|
||||
/mox
|
||||
|
||||
ESCAPE = /
|
||||
#{CONTROL_META_ESCAPE} | #{SIMPLE_ESCAPE}
|
||||
/mox
|
||||
|
||||
CHARACTER = /
|
||||
\?
|
||||
(?:
|
||||
[^\s\\]
|
||||
| \\ #{ESCAPE}
|
||||
)
|
||||
/mox
|
||||
|
||||
# NOTE: This is not completely correct, but
|
||||
# nobody needs heredoc delimiters ending with \n.
|
||||
HEREDOC_OPEN = /
|
||||
<< (-)? # $1 = float
|
||||
(?:
|
||||
( [A-Za-z_0-9]+ ) # $2 = delim
|
||||
|
|
||||
( ["'`\/] ) # $3 = quote, type
|
||||
( [^\n]*? ) \3 # $4 = delim
|
||||
)
|
||||
/mx
|
||||
|
||||
RUBYDOC = /
|
||||
=begin (?!\S)
|
||||
.*?
|
||||
(?: \Z | ^=end (?!\S) [^\n]* )
|
||||
/mx
|
||||
|
||||
DATA = /
|
||||
__END__$
|
||||
.*?
|
||||
(?: \Z | (?=^\#CODE) )
|
||||
/mx
|
||||
|
||||
RUBYDOC_OR_DATA = / #{RUBYDOC} | #{DATA} /xo
|
||||
|
||||
# Checks for a valid value to follow. This enables
|
||||
# value_expected in method calls without parentheses.
|
||||
VALUE_FOLLOWS = /
|
||||
(?>[ \t\f\v]+)
|
||||
(?:
|
||||
[%\/][^\s=]
|
||||
| <<-?\S
|
||||
| [-+] \d
|
||||
| #{CHARACTER}
|
||||
)
|
||||
/ox
|
||||
KEYWORDS_EXPECTING_VALUE = WordList.new.add(%w[
|
||||
and end in or unless begin
|
||||
defined? ensure redo super until
|
||||
break do next rescue then
|
||||
when case else for retry
|
||||
while elsif if not return
|
||||
yield
|
||||
])
|
||||
|
||||
FANCY_STRING_START = / % ( [QqrsWwx] | (?![a-zA-Z0-9]) ) ([^a-zA-Z0-9]) /x
|
||||
FANCY_STRING_KIND = Hash.new(:string).merge({
|
||||
'r' => :regexp,
|
||||
's' => :symbol,
|
||||
'x' => :shell,
|
||||
})
|
||||
FANCY_STRING_INTERPRETED = Hash.new(true).merge({
|
||||
'q' => false,
|
||||
's' => false,
|
||||
'w' => false,
|
||||
})
|
||||
|
||||
end
|
||||
|
||||
end
|
||||
end
|
||||
|
|
@ -0,0 +1,71 @@
|
|||
# encoding: utf-8
|
||||
module CodeRay
|
||||
module Scanners
|
||||
|
||||
class Ruby
|
||||
|
||||
class StringState < Struct.new :type, :interpreted, :delim, :heredoc,
|
||||
:opening_paren, :paren_depth, :pattern, :next_state # :nodoc: all
|
||||
|
||||
CLOSING_PAREN = Hash[ *%w[
|
||||
( )
|
||||
[ ]
|
||||
< >
|
||||
{ }
|
||||
] ].each { |k,v| k.freeze; v.freeze } # debug, if I try to change it with <<
|
||||
|
||||
STRING_PATTERN = Hash.new do |h, k|
|
||||
delim, interpreted = *k
|
||||
# delim = delim.dup # workaround for old Ruby
|
||||
delim_pattern = Regexp.escape(delim)
|
||||
if closing_paren = CLOSING_PAREN[delim]
|
||||
delim_pattern << Regexp.escape(closing_paren)
|
||||
end
|
||||
delim_pattern << '\\\\' unless delim == '\\'
|
||||
|
||||
# special_escapes =
|
||||
# case interpreted
|
||||
# when :regexp_symbols
|
||||
# '| [|?*+(){}\[\].^$]'
|
||||
# end
|
||||
|
||||
h[k] =
|
||||
if interpreted && delim != '#'
|
||||
/ (?= [#{delim_pattern}] | \# [{$@] ) /mx
|
||||
else
|
||||
/ (?= [#{delim_pattern}] ) /mx
|
||||
end
|
||||
end
|
||||
|
||||
def initialize kind, interpreted, delim, heredoc = false
|
||||
if heredoc
|
||||
pattern = heredoc_pattern delim, interpreted, heredoc == :indented
|
||||
delim = nil
|
||||
else
|
||||
pattern = STRING_PATTERN[ [delim, interpreted] ]
|
||||
if closing_paren = CLOSING_PAREN[delim]
|
||||
opening_paren = delim
|
||||
delim = closing_paren
|
||||
paren_depth = 1
|
||||
end
|
||||
end
|
||||
super kind, interpreted, delim, heredoc, opening_paren, paren_depth, pattern, :initial
|
||||
end
|
||||
|
||||
def heredoc_pattern delim, interpreted, indented
|
||||
# delim = delim.dup # workaround for old Ruby
|
||||
delim_pattern = Regexp.escape(delim)
|
||||
delim_pattern = / (?:\A|\n) #{ '(?>[ \t]*)' if indented } #{ Regexp.new delim_pattern } $ /x
|
||||
if interpreted
|
||||
/ (?= #{delim_pattern}() | \\ | \# [{$@] ) /mx # $1 set == end of heredoc
|
||||
else
|
||||
/ (?= #{delim_pattern}() | \\ ) /mx
|
||||
end
|
||||
end
|
||||
|
||||
end
|
||||
|
||||
end
|
||||
|
||||
end
|
||||
end
|
||||
|
|
@ -0,0 +1,174 @@
|
|||
module CodeRay module Scanners
|
||||
|
||||
# by Josh Goebel
|
||||
class SQL < Scanner
|
||||
|
||||
register_for :sql
|
||||
|
||||
KEYWORDS = %w(
|
||||
all and any as before begin between by case check collate
|
||||
each else end exists
|
||||
for foreign from full group having if in inner is join
|
||||
like not of on or order outer over references
|
||||
then to union using values when where
|
||||
left right distinct
|
||||
)
|
||||
|
||||
OBJECTS = %w(
|
||||
database databases table tables column columns fields index constraint
|
||||
constraints transaction function procedure row key view trigger
|
||||
)
|
||||
|
||||
COMMANDS = %w(
|
||||
add alter comment create delete drop grant insert into select update set
|
||||
show prompt begin commit rollback replace truncate
|
||||
)
|
||||
|
||||
PREDEFINED_TYPES = %w(
|
||||
char varchar varchar2 enum binary text tinytext mediumtext
|
||||
longtext blob tinyblob mediumblob longblob timestamp
|
||||
date time datetime year double decimal float int
|
||||
integer tinyint mediumint bigint smallint unsigned bit
|
||||
bool boolean hex bin oct
|
||||
)
|
||||
|
||||
PREDEFINED_FUNCTIONS = %w( sum cast substring abs pi count min max avg now )
|
||||
|
||||
DIRECTIVES = %w(
|
||||
auto_increment unique default charset initially deferred
|
||||
deferrable cascade immediate read write asc desc after
|
||||
primary foreign return engine
|
||||
)
|
||||
|
||||
PREDEFINED_CONSTANTS = %w( null true false )
|
||||
|
||||
IDENT_KIND = WordList::CaseIgnoring.new(:ident).
|
||||
add(KEYWORDS, :keyword).
|
||||
add(OBJECTS, :type).
|
||||
add(COMMANDS, :class).
|
||||
add(PREDEFINED_TYPES, :predefined_type).
|
||||
add(PREDEFINED_CONSTANTS, :predefined_constant).
|
||||
add(PREDEFINED_FUNCTIONS, :predefined).
|
||||
add(DIRECTIVES, :directive)
|
||||
|
||||
ESCAPE = / [rbfntv\n\\\/'"] | x[a-fA-F0-9]{1,2} | [0-7]{1,3} | . /mx
|
||||
UNICODE_ESCAPE = / u[a-fA-F0-9]{4} | U[a-fA-F0-9]{8} /x
|
||||
|
||||
STRING_PREFIXES = /[xnb]|_\w+/i
|
||||
|
||||
def scan_tokens encoder, options
|
||||
|
||||
state = :initial
|
||||
string_type = nil
|
||||
string_content = ''
|
||||
name_expected = false
|
||||
|
||||
until eos?
|
||||
|
||||
if state == :initial
|
||||
|
||||
if match = scan(/ \s+ | \\\n /x)
|
||||
encoder.text_token match, :space
|
||||
|
||||
elsif match = scan(/(?:--\s?|#).*/)
|
||||
encoder.text_token match, :comment
|
||||
|
||||
elsif match = scan(%r( /\* (!)? (?: .*? \*/ | .* ) )mx)
|
||||
encoder.text_token match, self[1] ? :directive : :comment
|
||||
|
||||
elsif match = scan(/ [*\/=<>:;,!&^|()\[\]{}~%] | [-+\.](?!\d) /x)
|
||||
name_expected = true if match == '.' && check(/[A-Za-z_]/)
|
||||
encoder.text_token match, :operator
|
||||
|
||||
elsif match = scan(/(#{STRING_PREFIXES})?([`"'])/o)
|
||||
prefix = self[1]
|
||||
string_type = self[2]
|
||||
encoder.begin_group :string
|
||||
encoder.text_token prefix, :modifier if prefix
|
||||
match = string_type
|
||||
state = :string
|
||||
encoder.text_token match, :delimiter
|
||||
|
||||
elsif match = scan(/ @? [A-Za-z_][A-Za-z_0-9]* /x)
|
||||
encoder.text_token match, name_expected ? :ident : (match[0] == ?@ ? :variable : IDENT_KIND[match])
|
||||
name_expected = false
|
||||
|
||||
elsif match = scan(/0[xX][0-9A-Fa-f]+/)
|
||||
encoder.text_token match, :hex
|
||||
|
||||
elsif match = scan(/0[0-7]+(?![89.eEfF])/)
|
||||
encoder.text_token match, :octal
|
||||
|
||||
elsif match = scan(/[-+]?(?>\d+)(?![.eEfF])/)
|
||||
encoder.text_token match, :integer
|
||||
|
||||
elsif match = scan(/[-+]?(?:\d[fF]|\d*\.\d+(?:[eE][+-]?\d+)?|\d+[eE][+-]?\d+)/)
|
||||
encoder.text_token match, :float
|
||||
|
||||
elsif match = scan(/\\N/)
|
||||
encoder.text_token match, :predefined_constant
|
||||
|
||||
else
|
||||
encoder.text_token getch, :error
|
||||
|
||||
end
|
||||
|
||||
elsif state == :string
|
||||
if match = scan(/[^\\"'`]+/)
|
||||
string_content << match
|
||||
next
|
||||
elsif match = scan(/["'`]/)
|
||||
if string_type == match
|
||||
if peek(1) == string_type # doubling means escape
|
||||
string_content << string_type << getch
|
||||
next
|
||||
end
|
||||
unless string_content.empty?
|
||||
encoder.text_token string_content, :content
|
||||
string_content = ''
|
||||
end
|
||||
encoder.text_token match, :delimiter
|
||||
encoder.end_group :string
|
||||
state = :initial
|
||||
string_type = nil
|
||||
else
|
||||
string_content << match
|
||||
end
|
||||
elsif match = scan(/ \\ (?: #{ESCAPE} | #{UNICODE_ESCAPE} ) /mox)
|
||||
unless string_content.empty?
|
||||
encoder.text_token string_content, :content
|
||||
string_content = ''
|
||||
end
|
||||
encoder.text_token match, :char
|
||||
elsif match = scan(/ \\ . /mox)
|
||||
string_content << match
|
||||
next
|
||||
elsif match = scan(/ \\ | $ /x)
|
||||
unless string_content.empty?
|
||||
encoder.text_token string_content, :content
|
||||
string_content = ''
|
||||
end
|
||||
encoder.text_token match, :error
|
||||
state = :initial
|
||||
else
|
||||
raise "else case \" reached; %p not handled." % peek(1), encoder
|
||||
end
|
||||
|
||||
else
|
||||
raise 'else-case reached', encoder
|
||||
|
||||
end
|
||||
|
||||
end
|
||||
|
||||
if state == :string
|
||||
encoder.end_group state
|
||||
end
|
||||
|
||||
encoder
|
||||
|
||||
end
|
||||
|
||||
end
|
||||
|
||||
end end
|
||||
|
|
@ -0,0 +1,26 @@
|
|||
module CodeRay
|
||||
module Scanners
|
||||
|
||||
# Scanner for plain text.
|
||||
#
|
||||
# Yields just one token of the kind :plain.
|
||||
#
|
||||
# Alias: +plaintext+, +plain+
|
||||
class Text < Scanner
|
||||
|
||||
register_for :text
|
||||
title 'Plain text'
|
||||
|
||||
KINDS_NOT_LOC = [:plain] # :nodoc:
|
||||
|
||||
protected
|
||||
|
||||
def scan_tokens encoder, options
|
||||
encoder.text_token string, :plain
|
||||
encoder
|
||||
end
|
||||
|
||||
end
|
||||
|
||||
end
|
||||
end
|
||||
|
|
@ -0,0 +1,17 @@
|
|||
module CodeRay
|
||||
module Scanners
|
||||
|
||||
load :html
|
||||
|
||||
# Scanner for XML.
|
||||
#
|
||||
# Currently this is the same scanner as Scanners::HTML.
|
||||
class XML < HTML
|
||||
|
||||
register_for :xml
|
||||
file_extension 'xml'
|
||||
|
||||
end
|
||||
|
||||
end
|
||||
end
|
||||
|
|
@ -0,0 +1,140 @@
|
|||
module CodeRay
|
||||
module Scanners
|
||||
|
||||
# Scanner for YAML.
|
||||
#
|
||||
# Based on the YAML scanner from Syntax by Jamis Buck.
|
||||
class YAML < Scanner
|
||||
|
||||
register_for :yaml
|
||||
file_extension 'yml'
|
||||
|
||||
KINDS_NOT_LOC = :all
|
||||
|
||||
protected
|
||||
|
||||
def scan_tokens encoder, options
|
||||
|
||||
state = :initial
|
||||
key_indent = string_indent = 0
|
||||
|
||||
until eos?
|
||||
|
||||
key_indent = nil if bol?
|
||||
|
||||
if match = scan(/ +[\t ]*/)
|
||||
encoder.text_token match, :space
|
||||
|
||||
elsif match = scan(/\n+/)
|
||||
encoder.text_token match, :space
|
||||
state = :initial if match.index(?\n)
|
||||
|
||||
elsif match = scan(/#.*/)
|
||||
encoder.text_token match, :comment
|
||||
|
||||
elsif bol? and case
|
||||
when match = scan(/---|\.\.\./)
|
||||
encoder.begin_group :head
|
||||
encoder.text_token match, :head
|
||||
encoder.end_group :head
|
||||
next
|
||||
when match = scan(/%.*/)
|
||||
encoder.text_token match, :doctype
|
||||
next
|
||||
end
|
||||
|
||||
elsif state == :value and case
|
||||
when !check(/(?:"[^"]*")(?=: |:$)/) && match = scan(/"/)
|
||||
encoder.begin_group :string
|
||||
encoder.text_token match, :delimiter
|
||||
encoder.text_token match, :content if match = scan(/ [^"\\]* (?: \\. [^"\\]* )* /mx)
|
||||
encoder.text_token match, :delimiter if match = scan(/"/)
|
||||
encoder.end_group :string
|
||||
next
|
||||
when match = scan(/[|>][-+]?/)
|
||||
encoder.begin_group :string
|
||||
encoder.text_token match, :delimiter
|
||||
string_indent = key_indent || column(pos - match.size) - 1
|
||||
encoder.text_token matched, :content if scan(/(?:\n+ {#{string_indent + 1}}.*)+/)
|
||||
encoder.end_group :string
|
||||
next
|
||||
when match = scan(/(?![!"*&]).+?(?=$|\s+#)/)
|
||||
encoder.begin_group :string
|
||||
encoder.text_token match, :content
|
||||
string_indent = key_indent || column(pos - match.size) - 1
|
||||
encoder.text_token matched, :content if scan(/(?:\n+ {#{string_indent + 1}}.*)+/)
|
||||
encoder.end_group :string
|
||||
next
|
||||
end
|
||||
|
||||
elsif case
|
||||
when match = scan(/[-:](?= |$)/)
|
||||
state = :value if state == :colon && (match == ':' || match == '-')
|
||||
state = :value if state == :initial && match == '-'
|
||||
encoder.text_token match, :operator
|
||||
next
|
||||
when match = scan(/[,{}\[\]]/)
|
||||
encoder.text_token match, :operator
|
||||
next
|
||||
when state == :initial && match = scan(/[-\w.()\/ ]*\S(?= *:(?: |$))/)
|
||||
encoder.text_token match, :key
|
||||
key_indent = column(pos - match.size) - 1
|
||||
state = :colon
|
||||
next
|
||||
when match = scan(/(?:"[^"\n]*"|'[^'\n]*')(?= *:(?: |$))/)
|
||||
encoder.begin_group :key
|
||||
encoder.text_token match[0,1], :delimiter
|
||||
encoder.text_token match[1..-2], :content
|
||||
encoder.text_token match[-1,1], :delimiter
|
||||
encoder.end_group :key
|
||||
key_indent = column(pos - match.size) - 1
|
||||
state = :colon
|
||||
next
|
||||
when match = scan(/(![\w\/]+)(:([\w:]+))?/)
|
||||
encoder.text_token self[1], :type
|
||||
if self[2]
|
||||
encoder.text_token ':', :operator
|
||||
encoder.text_token self[3], :class
|
||||
end
|
||||
next
|
||||
when match = scan(/&\S+/)
|
||||
encoder.text_token match, :variable
|
||||
next
|
||||
when match = scan(/\*\w+/)
|
||||
encoder.text_token match, :global_variable
|
||||
next
|
||||
when match = scan(/<</)
|
||||
encoder.text_token match, :class_variable
|
||||
next
|
||||
when match = scan(/\d\d:\d\d:\d\d/)
|
||||
encoder.text_token match, :octal
|
||||
next
|
||||
when match = scan(/\d\d\d\d-\d\d-\d\d\s\d\d:\d\d:\d\d(\.\d+)? [-+]\d\d:\d\d/)
|
||||
encoder.text_token match, :octal
|
||||
next
|
||||
when match = scan(/:\w+/)
|
||||
encoder.text_token match, :symbol
|
||||
next
|
||||
when match = scan(/[^:\s]+(:(?! |$)[^:\s]*)* .*/)
|
||||
encoder.text_token match, :error
|
||||
next
|
||||
when match = scan(/[^:\s]+(:(?! |$)[^:\s]*)*/)
|
||||
encoder.text_token match, :error
|
||||
next
|
||||
end
|
||||
|
||||
else
|
||||
raise if eos?
|
||||
encoder.text_token getch, :error
|
||||
|
||||
end
|
||||
|
||||
end
|
||||
|
||||
encoder
|
||||
end
|
||||
|
||||
end
|
||||
|
||||
end
|
||||
end
|
||||
|
|
@ -0,0 +1,23 @@
|
|||
module CodeRay
|
||||
|
||||
# This module holds the Style class and its subclasses.
|
||||
#
|
||||
# See Plugin.
|
||||
module Styles
|
||||
extend PluginHost
|
||||
plugin_path File.dirname(__FILE__), 'styles'
|
||||
|
||||
# Base class for styles.
|
||||
#
|
||||
# Styles are used by Encoders::HTML to colorize tokens.
|
||||
class Style
|
||||
extend Plugin
|
||||
plugin_host Styles
|
||||
|
||||
DEFAULT_OPTIONS = { } # :nodoc:
|
||||
|
||||
end
|
||||
|
||||
end
|
||||
|
||||
end
|
||||
|
|
@ -0,0 +1,7 @@
|
|||
module CodeRay
|
||||
module Styles
|
||||
|
||||
default :alpha
|
||||
|
||||
end
|
||||
end
|
||||
|
|
@ -0,0 +1,142 @@
|
|||
module CodeRay
|
||||
module Styles
|
||||
|
||||
# A colorful theme using CSS 3 colors (with alpha channel).
|
||||
class Alpha < Style
|
||||
|
||||
register_for :alpha
|
||||
|
||||
code_background = 'hsl(0,0%,95%)'
|
||||
numbers_background = 'hsl(180,65%,90%)'
|
||||
border_color = 'silver'
|
||||
normal_color = 'black'
|
||||
|
||||
CSS_MAIN_STYLES = <<-MAIN # :nodoc:
|
||||
.CodeRay {
|
||||
background-color: #{code_background};
|
||||
border: 1px solid #{border_color};
|
||||
color: #{normal_color};
|
||||
}
|
||||
.CodeRay pre {
|
||||
margin: 0px;
|
||||
}
|
||||
|
||||
span.CodeRay { white-space: pre; border: 0px; padding: 2px; }
|
||||
|
||||
table.CodeRay { border-collapse: collapse; width: 100%; padding: 2px; }
|
||||
table.CodeRay td { padding: 2px 4px; vertical-align: top; }
|
||||
|
||||
.CodeRay .line-numbers {
|
||||
background-color: #{numbers_background};
|
||||
color: gray;
|
||||
text-align: right;
|
||||
-webkit-user-select: none;
|
||||
-moz-user-select: none;
|
||||
user-select: none;
|
||||
}
|
||||
.CodeRay .line-numbers a {
|
||||
background-color: #{numbers_background} !important;
|
||||
color: gray !important;
|
||||
text-decoration: none !important;
|
||||
}
|
||||
.CodeRay .line-numbers a:target { color: blue !important; }
|
||||
.CodeRay .line-numbers .highlighted { color: red !important; }
|
||||
.CodeRay .line-numbers .highlighted a { color: red !important; }
|
||||
.CodeRay span.line-numbers { padding: 0px 4px; }
|
||||
.CodeRay .line { display: block; float: left; width: 100%; }
|
||||
.CodeRay .code { width: 100%; }
|
||||
MAIN
|
||||
|
||||
TOKEN_COLORS = <<-'TOKENS'
|
||||
.debug { color: white !important; background: blue !important; }
|
||||
|
||||
.annotation { color:#007 }
|
||||
.attribute-name { color:#b48 }
|
||||
.attribute-value { color:#700 }
|
||||
.binary { color:#509 }
|
||||
.char .content { color:#D20 }
|
||||
.char .delimiter { color:#710 }
|
||||
.char { color:#D20 }
|
||||
.class { color:#B06; font-weight:bold }
|
||||
.class-variable { color:#369 }
|
||||
.color { color:#0A0 }
|
||||
.comment { color:#777 }
|
||||
.comment .char { color:#444 }
|
||||
.comment .delimiter { color:#444 }
|
||||
.complex { color:#A08 }
|
||||
.constant { color:#036; font-weight:bold }
|
||||
.decorator { color:#B0B }
|
||||
.definition { color:#099; font-weight:bold }
|
||||
.delimiter { color:black }
|
||||
.directive { color:#088; font-weight:bold }
|
||||
.doc { color:#970 }
|
||||
.doc-string { color:#D42; font-weight:bold }
|
||||
.doctype { color:#34b }
|
||||
.entity { color:#800; font-weight:bold }
|
||||
.error { color:#F00; background-color:#FAA }
|
||||
.escape { color:#666 }
|
||||
.exception { color:#C00; font-weight:bold }
|
||||
.float { color:#60E }
|
||||
.function { color:#06B; font-weight:bold }
|
||||
.global-variable { color:#d70 }
|
||||
.hex { color:#02b }
|
||||
.imaginary { color:#f00 }
|
||||
.include { color:#B44; font-weight:bold }
|
||||
.inline { background-color: hsla(0,0%,0%,0.07); color: black }
|
||||
.inline-delimiter { font-weight: bold; color: #666 }
|
||||
.instance-variable { color:#33B }
|
||||
.integer { color:#00D }
|
||||
.key .char { color: #60f }
|
||||
.key .delimiter { color: #404 }
|
||||
.key { color: #606 }
|
||||
.keyword { color:#080; font-weight:bold }
|
||||
.label { color:#970; font-weight:bold }
|
||||
.local-variable { color:#963 }
|
||||
.namespace { color:#707; font-weight:bold }
|
||||
.octal { color:#40E }
|
||||
.operator { }
|
||||
.predefined { color:#369; font-weight:bold }
|
||||
.predefined-constant { color:#069 }
|
||||
.predefined-type { color:#0a5; font-weight:bold }
|
||||
.preprocessor { color:#579 }
|
||||
.pseudo-class { color:#00C; font-weight:bold }
|
||||
.regexp .content { color:#808 }
|
||||
.regexp .delimiter { color:#404 }
|
||||
.regexp .modifier { color:#C2C }
|
||||
.regexp { background-color:hsla(300,100%,50%,0.06); }
|
||||
.reserved { color:#080; font-weight:bold }
|
||||
.shell .content { color:#2B2 }
|
||||
.shell .delimiter { color:#161 }
|
||||
.shell { background-color:hsla(120,100%,50%,0.06); }
|
||||
.string .char { color: #b0b }
|
||||
.string .content { color: #D20 }
|
||||
.string .delimiter { color: #710 }
|
||||
.string .modifier { color: #E40 }
|
||||
.string { background-color:hsla(0,100%,50%,0.05); }
|
||||
.symbol .content { color:#A60 }
|
||||
.symbol .delimiter { color:#630 }
|
||||
.symbol { color:#A60 }
|
||||
.tag { color:#070 }
|
||||
.type { color:#339; font-weight:bold }
|
||||
.value { color: #088; }
|
||||
.variable { color:#037 }
|
||||
|
||||
.insert { background: hsla(120,100%,50%,0.12) }
|
||||
.delete { background: hsla(0,100%,50%,0.12) }
|
||||
.change { color: #bbf; background: #007; }
|
||||
.head { color: #f8f; background: #505 }
|
||||
.head .filename { color: white; }
|
||||
|
||||
.delete .eyecatcher { background-color: hsla(0,100%,50%,0.2); border: 1px solid hsla(0,100%,45%,0.5); margin: -1px; border-bottom: none; border-top-left-radius: 5px; border-top-right-radius: 5px; }
|
||||
.insert .eyecatcher { background-color: hsla(120,100%,50%,0.2); border: 1px solid hsla(120,100%,25%,0.5); margin: -1px; border-top: none; border-bottom-left-radius: 5px; border-bottom-right-radius: 5px; }
|
||||
|
||||
.insert .insert { color: #0c0; background:transparent; font-weight:bold }
|
||||
.delete .delete { color: #c00; background:transparent; font-weight:bold }
|
||||
.change .change { color: #88f }
|
||||
.head .head { color: #f4f }
|
||||
TOKENS
|
||||
|
||||
end
|
||||
|
||||
end
|
||||
end
|
||||
|
|
@ -0,0 +1,90 @@
|
|||
module CodeRay
|
||||
|
||||
# A Hash of all known token kinds and their associated CSS classes.
|
||||
TokenKinds = Hash.new do |h, k|
|
||||
warn 'Undefined Token kind: %p' % [k] if $CODERAY_DEBUG
|
||||
false
|
||||
end
|
||||
|
||||
# speedup
|
||||
TokenKinds.compare_by_identity if TokenKinds.respond_to? :compare_by_identity
|
||||
|
||||
TokenKinds.update( # :nodoc:
|
||||
:annotation => 'annotation',
|
||||
:attribute_name => 'attribute-name',
|
||||
:attribute_value => 'attribute-value',
|
||||
:binary => 'bin',
|
||||
:char => 'char',
|
||||
:class => 'class',
|
||||
:class_variable => 'class-variable',
|
||||
:color => 'color',
|
||||
:comment => 'comment',
|
||||
:complex => 'complex',
|
||||
:constant => 'constant',
|
||||
:content => 'content',
|
||||
:debug => 'debug',
|
||||
:decorator => 'decorator',
|
||||
:definition => 'definition',
|
||||
:delimiter => 'delimiter',
|
||||
:directive => 'directive',
|
||||
:doc => 'doc',
|
||||
:doctype => 'doctype',
|
||||
:doc_string => 'doc-string',
|
||||
:entity => 'entity',
|
||||
:error => 'error',
|
||||
:escape => 'escape',
|
||||
:exception => 'exception',
|
||||
:filename => 'filename',
|
||||
:float => 'float',
|
||||
:function => 'function',
|
||||
:global_variable => 'global-variable',
|
||||
:hex => 'hex',
|
||||
:imaginary => 'imaginary',
|
||||
:important => 'important',
|
||||
:include => 'include',
|
||||
:inline => 'inline',
|
||||
:inline_delimiter => 'inline-delimiter',
|
||||
:instance_variable => 'instance-variable',
|
||||
:integer => 'integer',
|
||||
:key => 'key',
|
||||
:keyword => 'keyword',
|
||||
:label => 'label',
|
||||
:local_variable => 'local-variable',
|
||||
:modifier => 'modifier',
|
||||
:namespace => 'namespace',
|
||||
:octal => 'octal',
|
||||
:predefined => 'predefined',
|
||||
:predefined_constant => 'predefined-constant',
|
||||
:predefined_type => 'predefined-type',
|
||||
:preprocessor => 'preprocessor',
|
||||
:pseudo_class => 'pseudo-class',
|
||||
:regexp => 'regexp',
|
||||
:reserved => 'reserved',
|
||||
:shell => 'shell',
|
||||
:string => 'string',
|
||||
:symbol => 'symbol',
|
||||
:tag => 'tag',
|
||||
:type => 'type',
|
||||
:value => 'value',
|
||||
:variable => 'variable',
|
||||
|
||||
:change => 'change',
|
||||
:delete => 'delete',
|
||||
:head => 'head',
|
||||
:insert => 'insert',
|
||||
|
||||
:eyecatcher => 'eyecatcher',
|
||||
|
||||
:ident => false,
|
||||
:operator => false,
|
||||
|
||||
:space => false,
|
||||
:plain => false
|
||||
)
|
||||
|
||||
TokenKinds[:method] = TokenKinds[:function]
|
||||
TokenKinds[:escape] = TokenKinds[:delimiter]
|
||||
TokenKinds[:docstring] = TokenKinds[:comment]
|
||||
|
||||
TokenKinds.freeze
|
||||
end
|
||||
|
|
@ -0,0 +1,215 @@
|
|||
module CodeRay
|
||||
|
||||
# GZip library for writing and reading token dumps.
|
||||
autoload :GZip, coderay_path('helpers', 'gzip')
|
||||
|
||||
# = Tokens TODO: Rewrite!
|
||||
#
|
||||
# The Tokens class represents a list of tokens returnd from
|
||||
# a Scanner.
|
||||
#
|
||||
# A token is not a special object, just a two-element Array
|
||||
# consisting of
|
||||
# * the _token_ _text_ (the original source of the token in a String) or
|
||||
# a _token_ _action_ (begin_group, end_group, begin_line, end_line)
|
||||
# * the _token_ _kind_ (a Symbol representing the type of the token)
|
||||
#
|
||||
# A token looks like this:
|
||||
#
|
||||
# ['# It looks like this', :comment]
|
||||
# ['3.1415926', :float]
|
||||
# ['$^', :error]
|
||||
#
|
||||
# Some scanners also yield sub-tokens, represented by special
|
||||
# token actions, namely begin_group and end_group.
|
||||
#
|
||||
# The Ruby scanner, for example, splits "a string" into:
|
||||
#
|
||||
# [
|
||||
# [:begin_group, :string],
|
||||
# ['"', :delimiter],
|
||||
# ['a string', :content],
|
||||
# ['"', :delimiter],
|
||||
# [:end_group, :string]
|
||||
# ]
|
||||
#
|
||||
# Tokens is the interface between Scanners and Encoders:
|
||||
# The input is split and saved into a Tokens object. The Encoder
|
||||
# then builds the output from this object.
|
||||
#
|
||||
# Thus, the syntax below becomes clear:
|
||||
#
|
||||
# CodeRay.scan('price = 2.59', :ruby).html
|
||||
# # the Tokens object is here -------^
|
||||
#
|
||||
# See how small it is? ;)
|
||||
#
|
||||
# Tokens gives you the power to handle pre-scanned code very easily:
|
||||
# You can convert it to a webpage, a YAML file, or dump it into a gzip'ed string
|
||||
# that you put in your DB.
|
||||
#
|
||||
# It also allows you to generate tokens directly (without using a scanner),
|
||||
# to load them from a file, and still use any Encoder that CodeRay provides.
|
||||
class Tokens < Array
|
||||
|
||||
# The Scanner instance that created the tokens.
|
||||
attr_accessor :scanner
|
||||
|
||||
# Encode the tokens using encoder.
|
||||
#
|
||||
# encoder can be
|
||||
# * a symbol like :html oder :statistic
|
||||
# * an Encoder class
|
||||
# * an Encoder object
|
||||
#
|
||||
# options are passed to the encoder.
|
||||
def encode encoder, options = {}
|
||||
encoder = Encoders[encoder].new options if encoder.respond_to? :to_sym
|
||||
encoder.encode_tokens self, options
|
||||
end
|
||||
|
||||
# Turn tokens into a string by concatenating them.
|
||||
def to_s
|
||||
encode CodeRay::Encoders::Encoder.new
|
||||
end
|
||||
|
||||
# Redirects unknown methods to encoder calls.
|
||||
#
|
||||
# For example, if you call +tokens.html+, the HTML encoder
|
||||
# is used to highlight the tokens.
|
||||
def method_missing meth, options = {}
|
||||
encode meth, options
|
||||
rescue PluginHost::PluginNotFound
|
||||
super
|
||||
end
|
||||
|
||||
# Split the tokens into parts of the given +sizes+.
|
||||
#
|
||||
# The result will be an Array of Tokens objects. The parts have
|
||||
# the text size specified by the parameter. In addition, each
|
||||
# part closes all opened tokens. This is useful to insert tokens
|
||||
# betweem them.
|
||||
#
|
||||
# This method is used by @Scanner#tokenize@ when called with an Array
|
||||
# of source strings. The Diff encoder uses it for inline highlighting.
|
||||
def split_into_parts *sizes
|
||||
parts = []
|
||||
opened = []
|
||||
content = nil
|
||||
part = Tokens.new
|
||||
part_size = 0
|
||||
size = sizes.first
|
||||
i = 0
|
||||
for item in self
|
||||
case content
|
||||
when nil
|
||||
content = item
|
||||
when String
|
||||
if size && part_size + content.size > size # token must be cut
|
||||
if part_size < size # some part of the token goes into this part
|
||||
content = content.dup # content may no be safe to change
|
||||
part << content.slice!(0, size - part_size) << item
|
||||
end
|
||||
# close all open groups and lines...
|
||||
closing = opened.reverse.flatten.map do |content_or_kind|
|
||||
case content_or_kind
|
||||
when :begin_group
|
||||
:end_group
|
||||
when :begin_line
|
||||
:end_line
|
||||
else
|
||||
content_or_kind
|
||||
end
|
||||
end
|
||||
part.concat closing
|
||||
begin
|
||||
parts << part
|
||||
part = Tokens.new
|
||||
size = sizes[i += 1]
|
||||
end until size.nil? || size > 0
|
||||
# ...and open them again.
|
||||
part.concat opened.flatten
|
||||
part_size = 0
|
||||
redo unless content.empty?
|
||||
else
|
||||
part << content << item
|
||||
part_size += content.size
|
||||
end
|
||||
content = nil
|
||||
when Symbol
|
||||
case content
|
||||
when :begin_group, :begin_line
|
||||
opened << [content, item]
|
||||
when :end_group, :end_line
|
||||
opened.pop
|
||||
else
|
||||
raise ArgumentError, 'Unknown token action: %p, kind = %p' % [content, item]
|
||||
end
|
||||
part << content << item
|
||||
content = nil
|
||||
else
|
||||
raise ArgumentError, 'Token input junk: %p, kind = %p' % [content, item]
|
||||
end
|
||||
end
|
||||
parts << part
|
||||
parts << Tokens.new while parts.size < sizes.size
|
||||
parts
|
||||
end
|
||||
|
||||
# Dumps the object into a String that can be saved
|
||||
# in files or databases.
|
||||
#
|
||||
# The dump is created with Marshal.dump;
|
||||
# In addition, it is gzipped using GZip.gzip.
|
||||
#
|
||||
# The returned String object includes Undumping
|
||||
# so it has an #undump method. See Tokens.load.
|
||||
#
|
||||
# You can configure the level of compression,
|
||||
# but the default value 7 should be what you want
|
||||
# in most cases as it is a good compromise between
|
||||
# speed and compression rate.
|
||||
#
|
||||
# See GZip module.
|
||||
def dump gzip_level = 7
|
||||
dump = Marshal.dump self
|
||||
dump = GZip.gzip dump, gzip_level
|
||||
dump.extend Undumping
|
||||
end
|
||||
|
||||
# Return the actual number of tokens.
|
||||
def count
|
||||
size / 2
|
||||
end
|
||||
|
||||
# Include this module to give an object an #undump
|
||||
# method.
|
||||
#
|
||||
# The string returned by Tokens.dump includes Undumping.
|
||||
module Undumping
|
||||
# Calls Tokens.load with itself.
|
||||
def undump
|
||||
Tokens.load self
|
||||
end
|
||||
end
|
||||
|
||||
# Undump the object using Marshal.load, then
|
||||
# unzip it using GZip.gunzip.
|
||||
#
|
||||
# The result is commonly a Tokens object, but
|
||||
# this is not guaranteed.
|
||||
def Tokens.load dump
|
||||
dump = GZip.gunzip dump
|
||||
@dump = Marshal.load dump
|
||||
end
|
||||
|
||||
alias text_token push
|
||||
def begin_group kind; push :begin_group, kind end
|
||||
def end_group kind; push :end_group, kind end
|
||||
def begin_line kind; push :begin_line, kind end
|
||||
def end_line kind; push :end_line, kind end
|
||||
alias tokens concat
|
||||
|
||||
end
|
||||
|
||||
end
|
||||
|
|
@ -0,0 +1,55 @@
|
|||
module CodeRay
|
||||
|
||||
# The result of a scan operation is a TokensProxy, but should act like Tokens.
|
||||
#
|
||||
# This proxy makes it possible to use the classic CodeRay.scan.encode API
|
||||
# while still providing the benefits of direct streaming.
|
||||
class TokensProxy
|
||||
|
||||
attr_accessor :input, :lang, :options, :block
|
||||
|
||||
# Create a new TokensProxy with the arguments of CodeRay.scan.
|
||||
def initialize input, lang, options = {}, block = nil
|
||||
@input = input
|
||||
@lang = lang
|
||||
@options = options
|
||||
@block = block
|
||||
end
|
||||
|
||||
# Call CodeRay.encode if +encoder+ is a Symbol;
|
||||
# otherwise, convert the receiver to tokens and call encoder.encode_tokens.
|
||||
def encode encoder, options = {}
|
||||
if encoder.respond_to? :to_sym
|
||||
CodeRay.encode(input, lang, encoder, options)
|
||||
else
|
||||
encoder.encode_tokens tokens, options
|
||||
end
|
||||
end
|
||||
|
||||
# Tries to call encode;
|
||||
# delegates to tokens otherwise.
|
||||
def method_missing method, *args, &blk
|
||||
encode method.to_sym, *args
|
||||
rescue PluginHost::PluginNotFound
|
||||
tokens.send(method, *args, &blk)
|
||||
end
|
||||
|
||||
# The (cached) result of the tokenized input; a Tokens instance.
|
||||
def tokens
|
||||
@tokens ||= scanner.tokenize(input)
|
||||
end
|
||||
|
||||
# A (cached) scanner instance to use for the scan task.
|
||||
def scanner
|
||||
@scanner ||= CodeRay.scanner(lang, options, &block)
|
||||
end
|
||||
|
||||
# Overwrite Struct#each.
|
||||
def each *args, &blk
|
||||
tokens.each(*args, &blk)
|
||||
self
|
||||
end
|
||||
|
||||
end
|
||||
|
||||
end
|
||||
|
|
@ -0,0 +1,3 @@
|
|||
module CodeRay
|
||||
VERSION = '1.0.7'
|
||||
end
|
||||
|
|
@ -0,0 +1,320 @@
|
|||
# encoding: utf-8
|
||||
require 'test/unit'
|
||||
require File.expand_path('../../lib/assert_warning', __FILE__)
|
||||
|
||||
$:.unshift File.expand_path('../../../lib', __FILE__)
|
||||
require 'coderay'
|
||||
|
||||
class BasicTest < Test::Unit::TestCase
|
||||
|
||||
def test_version
|
||||
assert_nothing_raised do
|
||||
assert_match(/\A\d\.\d\.\d?\z/, CodeRay::VERSION)
|
||||
end
|
||||
end
|
||||
|
||||
def with_empty_load_path
|
||||
old_load_path = $:.dup
|
||||
$:.clear
|
||||
yield
|
||||
ensure
|
||||
$:.replace old_load_path
|
||||
end
|
||||
|
||||
def test_autoload
|
||||
with_empty_load_path do
|
||||
assert_nothing_raised do
|
||||
CodeRay::Scanners::Java::BuiltinTypes
|
||||
end
|
||||
end
|
||||
end
|
||||
|
||||
RUBY_TEST_CODE = 'puts "Hello, World!"'
|
||||
|
||||
RUBY_TEST_TOKENS = [
|
||||
['puts', :ident],
|
||||
[' ', :space],
|
||||
[:begin_group, :string],
|
||||
['"', :delimiter],
|
||||
['Hello, World!', :content],
|
||||
['"', :delimiter],
|
||||
[:end_group, :string]
|
||||
].flatten
|
||||
def test_simple_scan
|
||||
assert_nothing_raised do
|
||||
assert_equal RUBY_TEST_TOKENS, CodeRay.scan(RUBY_TEST_CODE, :ruby).tokens
|
||||
end
|
||||
end
|
||||
|
||||
RUBY_TEST_HTML = 'puts <span class="string"><span class="delimiter">"</span>' +
|
||||
'<span class="content">Hello, World!</span><span class="delimiter">"</span></span>'
|
||||
def test_simple_highlight
|
||||
assert_nothing_raised do
|
||||
assert_equal RUBY_TEST_HTML, CodeRay.scan(RUBY_TEST_CODE, :ruby).html
|
||||
end
|
||||
end
|
||||
|
||||
def test_scan_file
|
||||
CodeRay.scan_file __FILE__
|
||||
end
|
||||
|
||||
def test_encode
|
||||
assert_equal 1, CodeRay.encode('test', :python, :count)
|
||||
end
|
||||
|
||||
def test_encode_tokens
|
||||
assert_equal 1, CodeRay.encode_tokens(CodeRay::Tokens['test', :string], :count)
|
||||
end
|
||||
|
||||
def test_encode_file
|
||||
assert_equal File.read(__FILE__), CodeRay.encode_file(__FILE__, :text)
|
||||
end
|
||||
|
||||
def test_highlight
|
||||
assert_match '<pre>test</pre>', CodeRay.highlight('test', :python)
|
||||
end
|
||||
|
||||
def test_highlight_file
|
||||
assert_match "require <span class=\"string\"><span class=\"delimiter\">'</span><span class=\"content\">test/unit</span><span class=\"delimiter\">'</span></span>\n", CodeRay.highlight_file(__FILE__)
|
||||
end
|
||||
|
||||
def test_duo
|
||||
assert_equal(RUBY_TEST_CODE,
|
||||
CodeRay::Duo[:plain, :text].highlight(RUBY_TEST_CODE))
|
||||
assert_equal(RUBY_TEST_CODE,
|
||||
CodeRay::Duo[:plain => :text].highlight(RUBY_TEST_CODE))
|
||||
end
|
||||
|
||||
def test_duo_stream
|
||||
assert_equal(RUBY_TEST_CODE,
|
||||
CodeRay::Duo[:plain, :text].highlight(RUBY_TEST_CODE, :stream => true))
|
||||
end
|
||||
|
||||
def test_comment_filter
|
||||
assert_equal <<-EXPECTED, CodeRay.scan(<<-INPUT, :ruby).comment_filter.text
|
||||
#!/usr/bin/env ruby
|
||||
|
||||
code
|
||||
|
||||
more code
|
||||
EXPECTED
|
||||
#!/usr/bin/env ruby
|
||||
=begin
|
||||
A multi-line comment.
|
||||
=end
|
||||
code
|
||||
# A single-line comment.
|
||||
more code # and another comment, in-line.
|
||||
INPUT
|
||||
end
|
||||
|
||||
def test_lines_of_code
|
||||
assert_equal 2, CodeRay.scan(<<-INPUT, :ruby).lines_of_code
|
||||
#!/usr/bin/env ruby
|
||||
=begin
|
||||
A multi-line comment.
|
||||
=end
|
||||
code
|
||||
# A single-line comment.
|
||||
more code # and another comment, in-line.
|
||||
INPUT
|
||||
rHTML = <<-RHTML
|
||||
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
|
||||
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
|
||||
|
||||
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
|
||||
<head>
|
||||
<meta http-equiv="content-type" content="text/html;charset=UTF-8" />
|
||||
<title><%= controller.controller_name.titleize %>: <%= controller.action_name %></title>
|
||||
<%= stylesheet_link_tag 'scaffold' %>
|
||||
</head>
|
||||
<body>
|
||||
|
||||
<p style="color: green"><%= flash[:notice] %></p>
|
||||
|
||||
<div id="main">
|
||||
<%= yield %>
|
||||
</div>
|
||||
|
||||
</body>
|
||||
</html>
|
||||
RHTML
|
||||
assert_equal 0, CodeRay.scan(rHTML, :html).lines_of_code
|
||||
assert_equal 0, CodeRay.scan(rHTML, :php).lines_of_code
|
||||
assert_equal 0, CodeRay.scan(rHTML, :yaml).lines_of_code
|
||||
assert_equal 4, CodeRay.scan(rHTML, :erb).lines_of_code
|
||||
end
|
||||
|
||||
def test_list_of_encoders
|
||||
assert_kind_of(Array, CodeRay::Encoders.list)
|
||||
assert CodeRay::Encoders.list.include?(:count)
|
||||
end
|
||||
|
||||
def test_list_of_scanners
|
||||
assert_kind_of(Array, CodeRay::Scanners.list)
|
||||
assert CodeRay::Scanners.list.include?(:text)
|
||||
end
|
||||
|
||||
def test_token_kinds
|
||||
assert_kind_of Hash, CodeRay::TokenKinds
|
||||
for kind, css_class in CodeRay::TokenKinds
|
||||
assert_kind_of Symbol, kind
|
||||
if css_class != false
|
||||
assert_kind_of String, css_class, "TokenKinds[%p] == %p" % [kind, css_class]
|
||||
end
|
||||
end
|
||||
assert_equal 'reserved', CodeRay::TokenKinds[:reserved]
|
||||
assert_warning 'Undefined Token kind: :shibboleet' do
|
||||
assert_equal false, CodeRay::TokenKinds[:shibboleet]
|
||||
end
|
||||
end
|
||||
|
||||
class Milk < CodeRay::Encoders::Encoder
|
||||
FILE_EXTENSION = 'cocoa'
|
||||
end
|
||||
|
||||
class HoneyBee < CodeRay::Encoders::Encoder
|
||||
end
|
||||
|
||||
def test_encoder_file_extension
|
||||
assert_nothing_raised do
|
||||
assert_equal 'html', CodeRay::Encoders::Page::FILE_EXTENSION
|
||||
assert_equal 'cocoa', Milk::FILE_EXTENSION
|
||||
assert_equal 'cocoa', Milk.new.file_extension
|
||||
assert_equal 'honeybee', HoneyBee::FILE_EXTENSION
|
||||
assert_equal 'honeybee', HoneyBee.new.file_extension
|
||||
end
|
||||
assert_raise NameError do
|
||||
HoneyBee::MISSING_CONSTANT
|
||||
end
|
||||
end
|
||||
|
||||
def test_encoder_tokens
|
||||
encoder = CodeRay::Encoders::Encoder.new
|
||||
encoder.send :setup, {}
|
||||
assert_raise(ArgumentError) { encoder.token :strange, '' }
|
||||
encoder.token 'test', :debug
|
||||
end
|
||||
|
||||
def test_encoder_deprecated_interface
|
||||
encoder = CodeRay::Encoders::Encoder.new
|
||||
encoder.send :setup, {}
|
||||
assert_warning 'Using old Tokens#<< interface.' do
|
||||
encoder << ['test', :content]
|
||||
end
|
||||
assert_raise ArgumentError do
|
||||
encoder << [:strange, :input]
|
||||
end
|
||||
assert_raise ArgumentError do
|
||||
encoder.encode_tokens [['test', :token]]
|
||||
end
|
||||
end
|
||||
|
||||
def encoder_token_interface_deprecation_warning_given
|
||||
CodeRay::Encoders::Encoder.send :class_variable_get, :@@CODERAY_TOKEN_INTERFACE_DEPRECATION_WARNING_GIVEN
|
||||
end
|
||||
|
||||
def test_scanner_file_extension
|
||||
assert_equal 'rb', CodeRay::Scanners::Ruby.file_extension
|
||||
assert_equal 'rb', CodeRay::Scanners::Ruby.new.file_extension
|
||||
assert_equal 'java', CodeRay::Scanners::Java.file_extension
|
||||
assert_equal 'java', CodeRay::Scanners::Java.new.file_extension
|
||||
end
|
||||
|
||||
def test_scanner_lang
|
||||
assert_equal :ruby, CodeRay::Scanners::Ruby.lang
|
||||
assert_equal :ruby, CodeRay::Scanners::Ruby.new.lang
|
||||
assert_equal :java, CodeRay::Scanners::Java.lang
|
||||
assert_equal :java, CodeRay::Scanners::Java.new.lang
|
||||
end
|
||||
|
||||
def test_scanner_tokenize
|
||||
assert_equal ['foo', :plain], CodeRay::Scanners::Plain.new.tokenize('foo')
|
||||
assert_equal [['foo', :plain], ['bar', :plain]], CodeRay::Scanners::Plain.new.tokenize(['foo', 'bar'])
|
||||
CodeRay::Scanners::Plain.new.tokenize 42
|
||||
end
|
||||
|
||||
def test_scanner_tokens
|
||||
scanner = CodeRay::Scanners::Plain.new
|
||||
scanner.tokenize('foo')
|
||||
assert_equal ['foo', :plain], scanner.tokens
|
||||
scanner.string = ''
|
||||
assert_equal ['', :plain], scanner.tokens
|
||||
end
|
||||
|
||||
def test_scanner_line_and_column
|
||||
scanner = CodeRay::Scanners::Plain.new "foo\nbär+quux"
|
||||
assert_equal 0, scanner.pos
|
||||
assert_equal 1, scanner.line
|
||||
assert_equal 1, scanner.column
|
||||
scanner.scan(/foo/)
|
||||
assert_equal 3, scanner.pos
|
||||
assert_equal 1, scanner.line
|
||||
assert_equal 4, scanner.column
|
||||
scanner.scan(/\n/)
|
||||
assert_equal 4, scanner.pos
|
||||
assert_equal 2, scanner.line
|
||||
assert_equal 1, scanner.column
|
||||
scanner.scan(/b/)
|
||||
assert_equal 5, scanner.pos
|
||||
assert_equal 2, scanner.line
|
||||
assert_equal 2, scanner.column
|
||||
scanner.scan(/a/)
|
||||
assert_equal 5, scanner.pos
|
||||
assert_equal 2, scanner.line
|
||||
assert_equal 2, scanner.column
|
||||
scanner.scan(/ä/)
|
||||
assert_equal 7, scanner.pos
|
||||
assert_equal 2, scanner.line
|
||||
assert_equal 4, scanner.column
|
||||
scanner.scan(/r/)
|
||||
assert_equal 8, scanner.pos
|
||||
assert_equal 2, scanner.line
|
||||
assert_equal 5, scanner.column
|
||||
end
|
||||
|
||||
def test_scanner_use_subclasses
|
||||
assert_raise NotImplementedError do
|
||||
CodeRay::Scanners::Scanner.new
|
||||
end
|
||||
end
|
||||
|
||||
class InvalidScanner < CodeRay::Scanners::Scanner
|
||||
end
|
||||
|
||||
def test_scanner_scan_tokens
|
||||
assert_raise NotImplementedError do
|
||||
InvalidScanner.new.tokenize ''
|
||||
end
|
||||
end
|
||||
|
||||
class RaisingScanner < CodeRay::Scanners::Scanner
|
||||
def scan_tokens encoder, options
|
||||
raise_inspect 'message', [], :initial
|
||||
end
|
||||
end
|
||||
|
||||
def test_scanner_raise_inspect
|
||||
assert_raise CodeRay::Scanners::Scanner::ScanError do
|
||||
RaisingScanner.new.tokenize ''
|
||||
end
|
||||
end
|
||||
|
||||
def test_scan_a_frozen_string
|
||||
assert_nothing_raised do
|
||||
CodeRay.scan RUBY_VERSION, :ruby
|
||||
CodeRay.scan RUBY_VERSION, :plain
|
||||
end
|
||||
end
|
||||
|
||||
def test_scan_a_non_string
|
||||
assert_nothing_raised do
|
||||
CodeRay.scan 42, :ruby
|
||||
CodeRay.scan nil, :ruby
|
||||
CodeRay.scan self, :ruby
|
||||
CodeRay.encode ENV.to_hash, :ruby, :page
|
||||
CodeRay.highlight CodeRay, :plain
|
||||
end
|
||||
end
|
||||
|
||||
end
|
||||
|
|
@ -0,0 +1,129 @@
|
|||
require 'test/unit'
|
||||
|
||||
$:.unshift File.expand_path('../../../lib', __FILE__)
|
||||
require 'coderay'
|
||||
|
||||
class ExamplesTest < Test::Unit::TestCase
|
||||
|
||||
def test_examples
|
||||
# output as HTML div (using inline CSS styles)
|
||||
div = CodeRay.scan('puts "Hello, world!"', :ruby).div
|
||||
assert_equal <<-DIV, div
|
||||
<div class="CodeRay">
|
||||
<div class="code"><pre>puts <span style="background-color:hsla(0,100%,50%,0.05)"><span style="color:#710">"</span><span style="color:#D20">Hello, world!</span><span style="color:#710">"</span></span></pre></div>
|
||||
</div>
|
||||
DIV
|
||||
|
||||
# ...with line numbers
|
||||
div = CodeRay.scan(<<-CODE.chomp, :ruby).div(:line_numbers => :table)
|
||||
5.times do
|
||||
puts 'Hello, world!'
|
||||
end
|
||||
CODE
|
||||
assert_equal <<-DIV, div
|
||||
<table class="CodeRay"><tr>
|
||||
<td class="line-numbers" title="double click to toggle" ondblclick="with (this.firstChild.style) { display = (display == '') ? 'none' : '' }"><pre><a href="#n1" name="n1">1</a>
|
||||
<a href="#n2" name="n2">2</a>
|
||||
<a href="#n3" name="n3">3</a>
|
||||
</pre></td>
|
||||
<td class="code"><pre><span style="color:#00D">5</span>.times <span style="color:#080;font-weight:bold">do</span>
|
||||
puts <span style="background-color:hsla(0,100%,50%,0.05)"><span style="color:#710">'</span><span style="color:#D20">Hello, world!</span><span style="color:#710">'</span></span>
|
||||
<span style="color:#080;font-weight:bold">end</span></pre></td>
|
||||
</tr></table>
|
||||
DIV
|
||||
|
||||
# output as standalone HTML page (using CSS classes)
|
||||
page = CodeRay.scan('puts "Hello, world!"', :ruby).page
|
||||
assert_match <<-PAGE, page
|
||||
<body>
|
||||
|
||||
<table class="CodeRay"><tr>
|
||||
<td class="line-numbers" title="double click to toggle" ondblclick="with (this.firstChild.style) { display = (display == '') ? 'none' : '' }"><pre>
|
||||
</pre></td>
|
||||
<td class="code"><pre>puts <span class="string"><span class="delimiter">"</span><span class="content">Hello, world!</span><span class="delimiter">"</span></span></pre></td>
|
||||
</tr></table>
|
||||
|
||||
</body>
|
||||
PAGE
|
||||
|
||||
# keep scanned tokens for later use
|
||||
tokens = CodeRay.scan('{ "just": "an", "example": 42 }', :json)
|
||||
assert_kind_of CodeRay::TokensProxy, tokens
|
||||
|
||||
assert_equal ["{", :operator, " ", :space, :begin_group, :key,
|
||||
"\"", :delimiter, "just", :content, "\"", :delimiter,
|
||||
:end_group, :key, ":", :operator, " ", :space,
|
||||
:begin_group, :string, "\"", :delimiter, "an", :content,
|
||||
"\"", :delimiter, :end_group, :string, ",", :operator,
|
||||
" ", :space, :begin_group, :key, "\"", :delimiter,
|
||||
"example", :content, "\"", :delimiter, :end_group, :key,
|
||||
":", :operator, " ", :space, "42", :integer,
|
||||
" ", :space, "}", :operator], tokens.tokens
|
||||
|
||||
# produce a token statistic
|
||||
assert_equal <<-STATISTIC, tokens.statistic
|
||||
|
||||
Code Statistics
|
||||
|
||||
Tokens 26
|
||||
Non-Whitespace 15
|
||||
Bytes Total 31
|
||||
|
||||
Token Types (7):
|
||||
type count ratio size (average)
|
||||
-------------------------------------------------------------
|
||||
TOTAL 26 100.00 % 1.2
|
||||
delimiter 6 23.08 % 1.0
|
||||
operator 5 19.23 % 1.0
|
||||
space 5 19.23 % 1.0
|
||||
key 4 15.38 % 0.0
|
||||
:begin_group 3 11.54 % 0.0
|
||||
:end_group 3 11.54 % 0.0
|
||||
content 3 11.54 % 4.3
|
||||
string 2 7.69 % 0.0
|
||||
integer 1 3.85 % 2.0
|
||||
|
||||
STATISTIC
|
||||
|
||||
# count the tokens
|
||||
assert_equal 26, tokens.count
|
||||
|
||||
# produce a HTML div, but with CSS classes
|
||||
div = tokens.div(:css => :class)
|
||||
assert_equal <<-DIV, div
|
||||
<div class="CodeRay">
|
||||
<div class="code"><pre>{ <span class="key"><span class="delimiter">"</span><span class="content">just</span><span class="delimiter">"</span></span>: <span class="string"><span class="delimiter">"</span><span class="content">an</span><span class="delimiter">"</span></span>, <span class="key"><span class="delimiter">"</span><span class="content">example</span><span class="delimiter">"</span></span>: <span class="integer">42</span> }</pre></div>
|
||||
</div>
|
||||
DIV
|
||||
|
||||
# highlight a file (HTML div); guess the file type base on the extension
|
||||
assert_equal :ruby, CodeRay::FileType[__FILE__]
|
||||
|
||||
# get a new scanner for Python
|
||||
python_scanner = CodeRay.scanner :python
|
||||
assert_kind_of CodeRay::Scanners::Python, python_scanner
|
||||
|
||||
# get a new encoder for terminal
|
||||
terminal_encoder = CodeRay.encoder :term
|
||||
assert_kind_of CodeRay::Encoders::Terminal, terminal_encoder
|
||||
|
||||
# scanning into tokens
|
||||
tokens = python_scanner.tokenize 'import this; # The Zen of Python'
|
||||
assert_equal ["import", :keyword, " ", :space, "this", :include,
|
||||
";", :operator, " ", :space, "# The Zen of Python", :comment], tokens
|
||||
|
||||
# format the tokens
|
||||
term = terminal_encoder.encode_tokens(tokens)
|
||||
assert_equal "\e[1;31mimport\e[0m \e[33mthis\e[0m; \e[37m# The Zen of Python\e[0m", term
|
||||
|
||||
# re-using scanner and encoder
|
||||
ruby_highlighter = CodeRay::Duo[:ruby, :div]
|
||||
div = ruby_highlighter.encode('puts "Hello, world!"')
|
||||
assert_equal <<-DIV, div
|
||||
<div class="CodeRay">
|
||||
<div class="code"><pre>puts <span style="background-color:hsla(0,100%,50%,0.05)"><span style="color:#710">"</span><span style="color:#D20">Hello, world!</span><span style="color:#710">"</span></span></pre></div>
|
||||
</div>
|
||||
DIV
|
||||
end
|
||||
|
||||
end
|
||||
|
|
@ -0,0 +1,84 @@
|
|||
require 'test/unit'
|
||||
require File.expand_path('../../lib/assert_warning', __FILE__)
|
||||
|
||||
$:.unshift File.expand_path('../../../lib', __FILE__)
|
||||
require 'coderay'
|
||||
|
||||
begin
|
||||
require 'rubygems' unless defined? Gem
|
||||
gem 'RedCloth', '>= 4.0.3' rescue nil
|
||||
require 'redcloth'
|
||||
rescue LoadError
|
||||
warn 'RedCloth not found - skipping for_redcloth tests.'
|
||||
undef RedCloth if defined? RedCloth
|
||||
end
|
||||
|
||||
class BasicTest < Test::Unit::TestCase
|
||||
|
||||
def test_for_redcloth
|
||||
require 'coderay/for_redcloth'
|
||||
assert_equal "<p><span lang=\"ruby\" class=\"CodeRay\">puts <span style=\"background-color:hsla(0,100%,50%,0.05)\"><span style=\"color:#710\">"</span><span style=\"color:#D20\">Hello, World!</span><span style=\"color:#710\">"</span></span></span></p>",
|
||||
RedCloth.new('@[ruby]puts "Hello, World!"@').to_html
|
||||
assert_equal <<-BLOCKCODE.chomp,
|
||||
<div lang="ruby" class="CodeRay">
|
||||
<div class="code"><pre>puts <span style="background-color:hsla(0,100%,50%,0.05)"><span style="color:#710">"</span><span style="color:#D20">Hello, World!</span><span style="color:#710">"</span></span></pre></div>
|
||||
</div>
|
||||
BLOCKCODE
|
||||
RedCloth.new('bc[ruby]. puts "Hello, World!"').to_html
|
||||
end
|
||||
|
||||
def test_for_redcloth_no_lang
|
||||
require 'coderay/for_redcloth'
|
||||
assert_equal "<p><code>puts \"Hello, World!\"</code></p>",
|
||||
RedCloth.new('@puts "Hello, World!"@').to_html
|
||||
assert_equal <<-BLOCKCODE.chomp,
|
||||
<pre><code>puts \"Hello, World!\"</code></pre>
|
||||
BLOCKCODE
|
||||
RedCloth.new('bc. puts "Hello, World!"').to_html
|
||||
end
|
||||
|
||||
def test_for_redcloth_style
|
||||
require 'coderay/for_redcloth'
|
||||
assert_equal <<-BLOCKCODE.chomp,
|
||||
<pre style=\"color: red;\"><code style=\"color: red;\">puts \"Hello, World!\"</code></pre>
|
||||
BLOCKCODE
|
||||
RedCloth.new('bc{color: red}. puts "Hello, World!"').to_html
|
||||
end
|
||||
|
||||
def test_for_redcloth_escapes
|
||||
require 'coderay/for_redcloth'
|
||||
assert_equal '<p><span lang="ruby" class="CodeRay">></span></p>',
|
||||
RedCloth.new('@[ruby]>@').to_html
|
||||
assert_equal <<-BLOCKCODE.chomp,
|
||||
<div lang="ruby" class="CodeRay">
|
||||
<div class="code"><pre>&</pre></div>
|
||||
</div>
|
||||
BLOCKCODE
|
||||
RedCloth.new('bc[ruby]. &').to_html
|
||||
end
|
||||
|
||||
def test_for_redcloth_escapes2
|
||||
require 'coderay/for_redcloth'
|
||||
assert_equal "<p><span lang=\"c\" class=\"CodeRay\"><span style=\"color:#579\">#include</span> <span style=\"color:#B44;font-weight:bold\"><test.h></span></span></p>",
|
||||
RedCloth.new('@[c]#include <test.h>@').to_html
|
||||
end
|
||||
|
||||
# See http://jgarber.lighthouseapp.com/projects/13054/tickets/124-code-markup-does-not-allow-brackets.
|
||||
def test_for_redcloth_false_positive
|
||||
require 'coderay/for_redcloth'
|
||||
assert_warning 'CodeRay::Scanners could not load plugin :project; falling back to :text' do
|
||||
assert_equal '<p><code>[project]_dff.skjd</code></p>',
|
||||
RedCloth.new('@[project]_dff.skjd@').to_html
|
||||
end
|
||||
# false positive, but expected behavior / known issue
|
||||
assert_equal "<p><span lang=\"ruby\" class=\"CodeRay\">_dff.skjd</span></p>",
|
||||
RedCloth.new('@[ruby]_dff.skjd@').to_html
|
||||
assert_warning 'CodeRay::Scanners could not load plugin :project; falling back to :text' do
|
||||
assert_equal <<-BLOCKCODE.chomp,
|
||||
<pre><code>[project]_dff.skjd</code></pre>
|
||||
BLOCKCODE
|
||||
RedCloth.new('bc. [project]_dff.skjd').to_html
|
||||
end
|
||||
end
|
||||
|
||||
end if defined? RedCloth
|
||||
|
|
@ -0,0 +1,15 @@
|
|||
require 'test/unit'
|
||||
|
||||
$VERBOSE = $CODERAY_DEBUG = true
|
||||
$:.unshift File.expand_path('../../../lib', __FILE__)
|
||||
require 'coderay'
|
||||
|
||||
mydir = File.dirname(__FILE__)
|
||||
suite = Dir[File.join(mydir, '*.rb')].
|
||||
map { |tc| File.basename(tc).sub(/\.rb$/, '') } - %w'suite for_redcloth'
|
||||
|
||||
puts "Running basic CodeRay #{CodeRay::VERSION} tests: #{suite.join(', ')}"
|
||||
|
||||
for test_case in suite
|
||||
load File.join(mydir, test_case + '.rb')
|
||||
end
|
||||
|
|
@ -0,0 +1,10 @@
|
|||
source :gemcutter
|
||||
gem 'cucumber', '~> 1.0.0'
|
||||
gem 'rspec', '~> 2.6'
|
||||
gem 'rake'
|
||||
|
||||
if RUBY_VERSION =~ /^1\.9/
|
||||
gem 'ruby-debug19'
|
||||
else
|
||||
gem 'ruby-debug'
|
||||
end
|
||||
|
|
@ -0,0 +1,20 @@
|
|||
Copyright (c) 2009 Tim Harper
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining
|
||||
a copy of this software and associated documentation files (the
|
||||
"Software"), to deal in the Software without restriction, including
|
||||
without limitation the rights to use, copy, modify, merge, publish,
|
||||
distribute, sublicense, and/or sell copies of the Software, and to
|
||||
permit persons to whom the Software is furnished to do so, subject to
|
||||
the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be
|
||||
included in all copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
|
||||
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
|
||||
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
|
||||
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
|
||||
LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
|
||||
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
|
||||
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
||||
|
|
@ -0,0 +1,134 @@
|
|||
= Spork
|
||||
|
||||
* Repository: http://github.com/sporkrb/spork
|
||||
* Issues: http://github.com/sporkrb/spork/issues
|
||||
* Changes: http://github.com/sporkrb/spork/blob/master/History.txt
|
||||
* Mailing list: http://groups.google.com/group/sporkgem
|
||||
* Wiki: http://wiki.github.com/sporkrb/spork
|
||||
|
||||
== SYNOPSIS:
|
||||
|
||||
Spork is Tim Harper's implementation of test server (similar to the script/spec_server that used to be provided by rspec-rails), except rather than using the Rails constant unloading to reload your files, it forks a copy of the server each time you run your tests. The result? Spork runs more solid: it doesn't get corrupted over time, it can work with any ruby framework, and it properly handles modules and any voodoo meta programming you may have put in your app.
|
||||
|
||||
Spork runs on POSIX systems using fork. It also runs on windows by pre-populating a pool of ready processes (referred to here as the "magazine" strategy).
|
||||
|
||||
== Supported Testing Frameworks
|
||||
|
||||
* Rspec
|
||||
* Cucumber
|
||||
* Test::Unit (via 'gem install spork-testunit')
|
||||
|
||||
== Supported Application Frameworks
|
||||
|
||||
Actually, Spork ~can~ work with any application framework. But, it ships with hooks and helpers to make an "out of the box" experience.
|
||||
|
||||
* Rails 3.0 (for Rails 2.{1,2,3}.x, please try spork 0.8.x)
|
||||
* Padrino
|
||||
|
||||
== INSTALL:
|
||||
|
||||
=== rubygems:
|
||||
|
||||
[sudo] gem install spork --version 0.9.0
|
||||
|
||||
(Rails 2.x)
|
||||
|
||||
[sudo] gem install spork --version 0.8.4
|
||||
|
||||
=== bundler:
|
||||
|
||||
Add to your Gemfile:
|
||||
|
||||
gem 'spork', '~> 0.9.0'
|
||||
|
||||
(Rails 2.x, use this)
|
||||
|
||||
gem 'spork', '~> 0.8'
|
||||
|
||||
== Usage
|
||||
|
||||
From a terminal, change to your project directory.
|
||||
|
||||
Then, bootstrap your spec/spec_helper.rb file.
|
||||
|
||||
spork --bootstrap
|
||||
|
||||
Next, edit spec/spec_helper.rb and follow the instructions that were put at the top.
|
||||
|
||||
Finally, run spork. A spec DRb server will be running!
|
||||
|
||||
spork
|
||||
|
||||
== Diagnostic mode
|
||||
|
||||
Initially, you may find that a few files don't reload automatically. This is because they are being loaded during Spork startup. To identify which project files are being pre-loaded, and why, run:
|
||||
|
||||
spork --diagnose | less
|
||||
(or spork -d, for short)
|
||||
|
||||
It will output a lot of stuff. At the top you'll find a summary of all project files loaded. Down below, the stack trace for each file (how it got loaded). Spork hooks into Rails and does some magic (TM) to prevent ApplicationController observers, etc from pre-loading. Similar hooks for other ruby frameworks may come as support demands.
|
||||
|
||||
Some project changes will require a spork restart, and you shouldn't expect them to not (IE: modifying your project gem dependencies or core application config). Any file that is loaded during Spork start-up will be cached until Spork is restarted. You can restart spork by sending a USR2 signal to the server process. This could be automated with a gem such as Kicker (https://github.com/alloy/kicker). Since a native hook is required for each operating system to efficiently watch for filesystem changes, automatic restarting has not been built into Spork.
|
||||
|
||||
You may find this wiki page useful for common ways to battle a variety of common pre-loading issues: http://github.com/sporkrb/spork/wiki/Spork.trap_method-Jujutsu
|
||||
|
||||
== Running specs over Spork
|
||||
|
||||
=== RSpec
|
||||
|
||||
To get the TextMate RSpec bundle to use spork, go to config->advanced->shell variables, and add:
|
||||
|
||||
TM_RSPEC_OPTS=--drb.
|
||||
|
||||
To run from the command line, use:
|
||||
|
||||
rspec --drb spec/lib/my_spec.rb
|
||||
|
||||
Or, you could add the following flag to your +spec.opts+ (or +.rspec+ depending on your version of rspec) file.
|
||||
|
||||
--drb
|
||||
|
||||
=== Cucumber
|
||||
|
||||
Cucumber has DRb support (see http://wiki.github.com/cucumber/cucumber/spork-and---drb)
|
||||
|
||||
cucumber --drb features/my.feature
|
||||
|
||||
Use this as a guideline when "Sporking" your features/support/env.rb file
|
||||
|
||||
http://gist.github.com/123370
|
||||
|
||||
== Running the Spork test suite
|
||||
|
||||
If you wish to hack on spork, you will want to run the automated test suite to make sure your changes don't break anything. Spork uses bundler to manage and install dependencies. To start:
|
||||
|
||||
bundle install
|
||||
|
||||
Then, to run the specs:
|
||||
|
||||
bundle exec rspec spec/
|
||||
|
||||
(or, alternatively...)
|
||||
|
||||
bundle exec rake
|
||||
|
||||
=== running features ===
|
||||
|
||||
Essentially:
|
||||
|
||||
bundle exec cucumber features
|
||||
|
||||
== Some potential issues and ways to overcome them:
|
||||
|
||||
See http://wiki.github.com/sporkrb/spork/troubleshooting
|
||||
|
||||
== Kudos to
|
||||
|
||||
* Ben Mabey - help with documentation, testing, suggestions, patches, and bringing Cucumber support.
|
||||
* David Chelimsky - for the fine RSpec testing framework, and the original rspec-rails spec_server implementation, which Spork has built upon.
|
||||
* LeadTune - just for being an awesome place to work.
|
||||
* Alan Aslak - for the fine Cucumber testing framework, and for much collaboration getting cucumber working with spork.
|
||||
* Roger Pack - JRuby support / Windows
|
||||
* Donald Parish - Windows support (Magazine strategy)
|
||||
|
||||
Spork (c) 2011 Tim Harper, released under the MIT license
|
||||
|
|
@ -0,0 +1,49 @@
|
|||
require 'rubygems'
|
||||
require 'spork'
|
||||
#uncomment the following line to use spork with the debugger
|
||||
#require 'spork/ext/ruby-debug'
|
||||
|
||||
Spork.prefork do
|
||||
# Loading more in this block will cause your tests to run faster. However,
|
||||
# if you change any configuration or code from libraries loaded here, you'll
|
||||
# need to restart spork for it take effect.
|
||||
|
||||
end
|
||||
|
||||
Spork.each_run do
|
||||
# This code will be run each time you run your specs.
|
||||
|
||||
end
|
||||
|
||||
# --- Instructions ---
|
||||
# Sort the contents of this file into a Spork.prefork and a Spork.each_run
|
||||
# block.
|
||||
#
|
||||
# The Spork.prefork block is run only once when the spork server is started.
|
||||
# You typically want to place most of your (slow) initializer code in here, in
|
||||
# particular, require'ing any 3rd-party gems that you don't normally modify
|
||||
# during development.
|
||||
#
|
||||
# The Spork.each_run block is run each time you run your specs. In case you
|
||||
# need to load files that tend to change during development, require them here.
|
||||
# With Rails, your application modules are loaded automatically, so sometimes
|
||||
# this block can remain empty.
|
||||
#
|
||||
# Note: You can modify files loaded *from* the Spork.each_run block without
|
||||
# restarting the spork server. However, this file itself will not be reloaded,
|
||||
# so if you change any of the code inside the each_run block, you still need to
|
||||
# restart the server. In general, if you have non-trivial code in this file,
|
||||
# it's advisable to move it into a separate file so you can easily edit it
|
||||
# without restarting spork. (For example, with RSpec, you could move
|
||||
# non-trivial code into a file spec/support/my_helper.rb, making sure that the
|
||||
# spec/support/* files are require'd from inside the each_run block.)
|
||||
#
|
||||
# Any code that is left outside the two blocks will be run during preforking
|
||||
# *and* during each_run -- that's probably not what you want.
|
||||
#
|
||||
# These instructions should self-destruct in 10 seconds. If they don't, feel
|
||||
# free to delete them.
|
||||
|
||||
|
||||
|
||||
|
||||
|
|
@ -0,0 +1,20 @@
|
|||
#!/usr/bin/env ruby
|
||||
require 'rubygems'
|
||||
|
||||
$LOAD_PATH.unshift(File.dirname(__FILE__) + '/../lib') unless $LOAD_PATH.include?(File.dirname(__FILE__) + '/../lib')
|
||||
|
||||
require 'spork'
|
||||
require 'spork/runner'
|
||||
|
||||
begin
|
||||
success = Spork::Runner.run(ARGV, STDOUT, STDERR)
|
||||
Kernel.exit(success ? 0 : 1)
|
||||
rescue SystemExit => e
|
||||
Kernel.exit(e.status)
|
||||
rescue Exception => e
|
||||
STDERR.puts("#{e.message} (#{e.class})")
|
||||
STDERR.puts(e.backtrace.join("\n"))
|
||||
Kernel.exit 1
|
||||
end
|
||||
|
||||
|
||||
|
|
@ -0,0 +1,36 @@
|
|||
Feature: At exit during each run
|
||||
In order to make sure at_exit hooks defined during the run get called
|
||||
I want to override kernel #at_exit
|
||||
|
||||
Scenario: at exit
|
||||
|
||||
Given a file named "spec/spec_helper.rb" with:
|
||||
"""
|
||||
require 'rubygems'
|
||||
require 'rspec'
|
||||
Spork.prefork do
|
||||
puts "loading"
|
||||
at_exit { puts "prefork at_exit called" }
|
||||
end
|
||||
|
||||
Spork.each_run do
|
||||
puts "running"
|
||||
at_exit { printf "first " }
|
||||
at_exit { printf "second " }
|
||||
end
|
||||
|
||||
"""
|
||||
|
||||
And a file named "spec/did_it_work_spec.rb" with:
|
||||
"""
|
||||
require 'spec_helper'
|
||||
describe "Did it work?" do
|
||||
it "checks to see if all worked" do
|
||||
puts "ran specs"
|
||||
end
|
||||
end
|
||||
"""
|
||||
When I fire up a spork instance with "spork rspec"
|
||||
And I run rspec --drb spec/did_it_work_spec.rb
|
||||
Then the output should contain "second first"
|
||||
Then the output should not contain "prefork at_exit called"
|
||||
|
|
@ -0,0 +1,107 @@
|
|||
Feature: Cucumber integration with rails
|
||||
As a developer using cucumber and rails
|
||||
I want to use Spork with Cucumber
|
||||
In order to eliminate the startup cost of my application each time I run them
|
||||
|
||||
Background: Sporked env.rb
|
||||
Given I am in a fresh rails project named "test_rails_project"
|
||||
And the application has a model, observer, route, and application helper
|
||||
And a file named "features/support/env.rb" with:
|
||||
"""
|
||||
require 'rubygems'
|
||||
require 'spork'
|
||||
ENV["RAILS_ENV"] ||= "test"
|
||||
|
||||
|
||||
Spork.prefork do
|
||||
# Loading more in this block will cause your tests to run faster. However,
|
||||
# if you change any configuration or code from libraries loaded here, you'll
|
||||
# need to restart spork for it take effect.
|
||||
|
||||
# Sets up the Rails environment for Cucumber
|
||||
require File.expand_path(File.dirname(__FILE__) + '/../../config/environment')
|
||||
|
||||
require 'cucumber'
|
||||
require 'cucumber/formatter/unicode' # Comment out this line if you don't want Cucumber Unicode support
|
||||
require 'rspec/rails'
|
||||
require 'cucumber/rails/rspec'
|
||||
|
||||
#### this is for this test only #######
|
||||
$loaded_stuff << 'prefork block' ######
|
||||
#######################################
|
||||
end
|
||||
|
||||
Spork.each_run do
|
||||
#### this is for this test only #######
|
||||
$loaded_stuff << 'each_run block' #####
|
||||
#######################################
|
||||
end
|
||||
"""
|
||||
And a file named "features/cucumber_rails.feature" with:
|
||||
"""
|
||||
Feature: cucumber rails
|
||||
Scenario: did it work?
|
||||
Then it should work
|
||||
|
||||
Scenario: did it work again?
|
||||
Then it should work
|
||||
"""
|
||||
And a file named "features/support/cucumber_rails_helper.rb" with:
|
||||
"""
|
||||
$loaded_stuff << 'features/support/cucumber_rails_helper.rb'
|
||||
"""
|
||||
And a file named "config/database.yml" with:
|
||||
"""
|
||||
test:
|
||||
adapter: sqlite3
|
||||
database: db/test.sqlite3
|
||||
timeout: 5000
|
||||
"""
|
||||
And a file named "features/step_definitions/cucumber_rails_steps.rb" with:
|
||||
"""
|
||||
Then "it should work" do
|
||||
(Rails.respond_to?(:logger) ? Rails.logger : ActionController::Base.logger).info "hey there"
|
||||
$loaded_stuff.should include('ActiveRecord::Base.establish_connection')
|
||||
$loaded_stuff.should include('User')
|
||||
$loaded_stuff.should include('UserObserver')
|
||||
$loaded_stuff.should include('ApplicationHelper')
|
||||
$loaded_stuff.should include('config/routes.rb')
|
||||
$loaded_stuff.should include('features/support/cucumber_rails_helper.rb')
|
||||
$loaded_stuff.should include('each_run block')
|
||||
$loaded_stuff.should include('prefork block')
|
||||
puts "It worked!"
|
||||
end
|
||||
|
||||
Alors /ca marche/ do
|
||||
end
|
||||
"""
|
||||
Scenario: Analyzing files were preloaded
|
||||
When I run spork --diagnose
|
||||
Then the output should not contain "user_observer.rb"
|
||||
Then the output should not contain "user.rb"
|
||||
Then the output should not contain "app/controllers/application.rb"
|
||||
Then the output should not contain "app/controllers/application_controller.rb"
|
||||
Then the output should not contain "app/controllers/application_helper.rb"
|
||||
Then the output should not contain "features/step_definitions/cucumber_rails_steps.rb"
|
||||
Then the output should not contain "features/support/cucumber_rails_helper.rb"
|
||||
|
||||
Scenario: Running spork with a rails app and no server
|
||||
When I run cucumber --drb features
|
||||
Then the error output should contain
|
||||
"""
|
||||
WARNING: No DRb server is running. Running features locally
|
||||
"""
|
||||
|
||||
Scenario: Running spork with a rails app and observers
|
||||
When I fire up a spork instance with "spork cucumber"
|
||||
And I run cucumber --drb features
|
||||
Then the error output should be empty
|
||||
And the output should contain "It worked!"
|
||||
And the file "log/test.log" should include "hey there"
|
||||
|
||||
Scenario: Running spork with a rails app and a non-standard port
|
||||
When I fire up a spork instance with "spork cucumber -p 9000"
|
||||
And I run cucumber --drb --port 9000 features
|
||||
Then the error output should be empty
|
||||
And the output should contain "It worked!"
|
||||
And the file "log/test.log" should include "hey there"
|
||||
|
|
@ -0,0 +1,41 @@
|
|||
Feature: Diagnostic Mode
|
||||
To help a developer quickly pinpoint why files are being loaded
|
||||
Spork provides a diagnostic mode
|
||||
That provides a list of which project files were loaded during prefork, and who loaded them.
|
||||
|
||||
Scenario: Running spork --diagnose
|
||||
Given I am in the directory "test_project"
|
||||
And a file named "spec/spec_helper.rb" with:
|
||||
"""
|
||||
require 'rubygems'
|
||||
require 'spork'
|
||||
|
||||
Spork.prefork do
|
||||
require 'lib/awesome.rb'
|
||||
require '../external_dependency/super_duper.rb'
|
||||
end
|
||||
|
||||
Spork.each_run do
|
||||
puts "I'm loading the stuff just for this run..."
|
||||
end
|
||||
"""
|
||||
And a file named "lib/awesome.rb" with:
|
||||
"""
|
||||
class Awesome
|
||||
end
|
||||
"""
|
||||
And a file named "../external_dependency/super_duper.rb" with:
|
||||
"""
|
||||
class Awesome
|
||||
end
|
||||
"""
|
||||
When I run spork --diagnose
|
||||
Then the error output should contain
|
||||
"""
|
||||
Loading Spork.prefork block...
|
||||
"""
|
||||
And the output should contain "lib/awesome.rb"
|
||||
And the output should contain "spec/spec_helper.rb:5"
|
||||
And the output should not contain "super_duper.rb"
|
||||
And the output should not contain "diagnose.rb"
|
||||
|
||||
|
|
@ -0,0 +1,14 @@
|
|||
source :gemcutter
|
||||
gem 'sqlite3-ruby', '1.2.5'
|
||||
gem 'cucumber', '~> 1.0.0'
|
||||
gem 'cucumber-rails', '~> 1.0.0'
|
||||
gem "rspec", "2.5.0"
|
||||
gem 'rspec-rails', "2.5.0"
|
||||
gem 'rails', '3.0.7'
|
||||
|
||||
if RUBY_VERSION =~ /^1\.9/
|
||||
gem 'ruby-debug19'
|
||||
else
|
||||
gem 'ruby-debug', '>= 0.10.3'
|
||||
end
|
||||
gem 'spork', :path => File.expand_path("../../..", File.dirname(__FILE__))
|
||||
|
|
@ -0,0 +1,139 @@
|
|||
PATH
|
||||
remote: /Users/timcharper/projects/spork
|
||||
specs:
|
||||
spork (0.9.0.rc8)
|
||||
|
||||
GEM
|
||||
remote: http://rubygems.org/
|
||||
specs:
|
||||
abstract (1.0.0)
|
||||
actionmailer (3.0.7)
|
||||
actionpack (= 3.0.7)
|
||||
mail (~> 2.2.15)
|
||||
actionpack (3.0.7)
|
||||
activemodel (= 3.0.7)
|
||||
activesupport (= 3.0.7)
|
||||
builder (~> 2.1.2)
|
||||
erubis (~> 2.6.6)
|
||||
i18n (~> 0.5.0)
|
||||
rack (~> 1.2.1)
|
||||
rack-mount (~> 0.6.14)
|
||||
rack-test (~> 0.5.7)
|
||||
tzinfo (~> 0.3.23)
|
||||
activemodel (3.0.7)
|
||||
activesupport (= 3.0.7)
|
||||
builder (~> 2.1.2)
|
||||
i18n (~> 0.5.0)
|
||||
activerecord (3.0.7)
|
||||
activemodel (= 3.0.7)
|
||||
activesupport (= 3.0.7)
|
||||
arel (~> 2.0.2)
|
||||
tzinfo (~> 0.3.23)
|
||||
activeresource (3.0.7)
|
||||
activemodel (= 3.0.7)
|
||||
activesupport (= 3.0.7)
|
||||
activesupport (3.0.7)
|
||||
arel (2.0.9)
|
||||
builder (2.1.2)
|
||||
capybara (1.0.0)
|
||||
mime-types (>= 1.16)
|
||||
nokogiri (>= 1.3.3)
|
||||
rack (>= 1.0.0)
|
||||
rack-test (>= 0.5.4)
|
||||
selenium-webdriver (~> 0.2.0)
|
||||
xpath (~> 0.1.4)
|
||||
childprocess (0.1.9)
|
||||
ffi (~> 1.0.6)
|
||||
columnize (0.3.2)
|
||||
cucumber (1.0.0)
|
||||
builder (>= 2.1.2)
|
||||
diff-lcs (>= 1.1.2)
|
||||
gherkin (~> 2.4.1)
|
||||
json (>= 1.4.6)
|
||||
term-ansicolor (>= 1.0.5)
|
||||
cucumber-rails (1.0.2)
|
||||
capybara (>= 1.0.0)
|
||||
cucumber (~> 1.0.0)
|
||||
nokogiri (>= 1.4.6)
|
||||
diff-lcs (1.1.2)
|
||||
erubis (2.6.6)
|
||||
abstract (>= 1.0.0)
|
||||
ffi (1.0.9)
|
||||
gherkin (2.4.1)
|
||||
json (>= 1.4.6)
|
||||
i18n (0.5.0)
|
||||
json (1.5.3)
|
||||
json_pure (1.5.3)
|
||||
linecache (0.43)
|
||||
mail (2.2.19)
|
||||
activesupport (>= 2.3.6)
|
||||
i18n (>= 0.4.0)
|
||||
mime-types (~> 1.16)
|
||||
treetop (~> 1.4.8)
|
||||
mime-types (1.16)
|
||||
nokogiri (1.4.6)
|
||||
polyglot (0.3.1)
|
||||
rack (1.2.2)
|
||||
rack-mount (0.6.14)
|
||||
rack (>= 1.0.0)
|
||||
rack-test (0.5.7)
|
||||
rack (>= 1.0)
|
||||
rails (3.0.7)
|
||||
actionmailer (= 3.0.7)
|
||||
actionpack (= 3.0.7)
|
||||
activerecord (= 3.0.7)
|
||||
activeresource (= 3.0.7)
|
||||
activesupport (= 3.0.7)
|
||||
bundler (~> 1.0)
|
||||
railties (= 3.0.7)
|
||||
railties (3.0.7)
|
||||
actionpack (= 3.0.7)
|
||||
activesupport (= 3.0.7)
|
||||
rake (>= 0.8.7)
|
||||
thor (~> 0.14.4)
|
||||
rake (0.8.7)
|
||||
rspec (2.5.0)
|
||||
rspec-core (~> 2.5.0)
|
||||
rspec-expectations (~> 2.5.0)
|
||||
rspec-mocks (~> 2.5.0)
|
||||
rspec-core (2.5.2)
|
||||
rspec-expectations (2.5.0)
|
||||
diff-lcs (~> 1.1.2)
|
||||
rspec-mocks (2.5.0)
|
||||
rspec-rails (2.5.0)
|
||||
actionpack (~> 3.0)
|
||||
activesupport (~> 3.0)
|
||||
railties (~> 3.0)
|
||||
rspec (~> 2.5.0)
|
||||
ruby-debug (0.10.4)
|
||||
columnize (>= 0.1)
|
||||
ruby-debug-base (~> 0.10.4.0)
|
||||
ruby-debug-base (0.10.4)
|
||||
linecache (>= 0.3)
|
||||
rubyzip (0.9.4)
|
||||
selenium-webdriver (0.2.2)
|
||||
childprocess (>= 0.1.9)
|
||||
ffi (>= 1.0.7)
|
||||
json_pure
|
||||
rubyzip
|
||||
sqlite3-ruby (1.2.5)
|
||||
term-ansicolor (1.0.5)
|
||||
thor (0.14.6)
|
||||
treetop (1.4.9)
|
||||
polyglot (>= 0.3.1)
|
||||
tzinfo (0.3.27)
|
||||
xpath (0.1.4)
|
||||
nokogiri (~> 1.3)
|
||||
|
||||
PLATFORMS
|
||||
ruby
|
||||
|
||||
DEPENDENCIES
|
||||
cucumber (~> 1.0.0)
|
||||
cucumber-rails (~> 1.0.0)
|
||||
rails (= 3.0.7)
|
||||
rspec (= 2.5.0)
|
||||
rspec-rails (= 2.5.0)
|
||||
ruby-debug (>= 0.10.3)
|
||||
spork!
|
||||
sqlite3-ruby (= 1.2.5)
|
||||
|
|
@ -0,0 +1,177 @@
|
|||
Feature: Rails Delayed Work arounds
|
||||
To allow a rails developer to update as many parts of his application as possible without needing to restart Spork
|
||||
Spork automatically tells rails to delay loading certain parts of the application until after the fork occurs
|
||||
|
||||
Background: Rails App with RSpec and Spork
|
||||
|
||||
Given I am in a fresh rails project named "test_rails_project"
|
||||
And a file named "spec/spec_helper.rb" with:
|
||||
"""
|
||||
require 'rubygems'
|
||||
require 'spork'
|
||||
require 'spork/ext/ruby-debug'
|
||||
|
||||
Spork.prefork do
|
||||
require File.dirname(__FILE__) + '/../config/environment.rb'
|
||||
require 'rspec'
|
||||
require 'rspec/rails'
|
||||
end
|
||||
|
||||
Spork.each_run do
|
||||
end
|
||||
"""
|
||||
And the application has a model, observer, route, and application helper
|
||||
|
||||
Given the following code appears in "config/routes.rb" after /routes\.draw/:
|
||||
"""
|
||||
resources :users
|
||||
"""
|
||||
Given a file named "app/helpers/application_helper.rb" with:
|
||||
"""
|
||||
require 'reverseatron'
|
||||
module ApplicationHelper
|
||||
include Reverseatron
|
||||
end
|
||||
"""
|
||||
Given a file named "lib/reverseatron.rb" with:
|
||||
"""
|
||||
module Reverseatron
|
||||
def reverse_text(txt)
|
||||
txt.reverse
|
||||
end
|
||||
end
|
||||
"""
|
||||
Given a file named "app/controllers/users_controller.rb" with:
|
||||
"""
|
||||
class UsersController < ApplicationController
|
||||
$loaded_stuff << 'UsersController'
|
||||
def index
|
||||
@users = []
|
||||
end
|
||||
end
|
||||
"""
|
||||
Given a file named "app/helpers/misc_helper.rb" with:
|
||||
"""
|
||||
module MiscHelper
|
||||
def misc_helper_method
|
||||
'hello miscellaneous'
|
||||
end
|
||||
end
|
||||
"""
|
||||
Given a file named "app/helpers/users_helper.rb" with:
|
||||
"""
|
||||
module UsersHelper
|
||||
end
|
||||
"""
|
||||
Given a file named "app/views/users/index.html.erb" with:
|
||||
"""
|
||||
Original View
|
||||
"""
|
||||
Scenario: respecting custom autoload paths
|
||||
Given the following code appears in "config/application.rb" after /class Application < Rails::Application/:
|
||||
"""
|
||||
config.autoload_paths << 'app/models/non_standard'
|
||||
"""
|
||||
|
||||
And a file named "app/models/non_standard/boogie.rb" with:
|
||||
"""
|
||||
class Boogie
|
||||
def boogie
|
||||
'Boogie Robots!'
|
||||
end
|
||||
end
|
||||
"""
|
||||
And a file named "spec/models/non_standard/boogie_spec.rb" with:
|
||||
"""
|
||||
describe Boogie do
|
||||
it 'knows how to boogie' do
|
||||
Boogie.new.boogie.should include('Boogie')
|
||||
puts 'BOOGIE!!!'
|
||||
end
|
||||
end
|
||||
"""
|
||||
When I fire up a spork instance with "spork rspec"
|
||||
And I run rspec --drb spec/models/non_standard/boogie_spec.rb
|
||||
Then the output should contain "BOOGIE!!!"
|
||||
|
||||
Scenario: within a view rendered by a controller, calling helper methods from an included module in ApplicationHelper
|
||||
Given a file named "spec/controllers/users_controller_spec.rb" with:
|
||||
"""
|
||||
require "spec_helper"
|
||||
describe UsersController do
|
||||
render_views
|
||||
it "renders a page, using a method inherited from ApplicationController" do
|
||||
get :index
|
||||
response.body.should_not include('Original View')
|
||||
puts "Views are not being cached when rendering from a controller"
|
||||
|
||||
response.body.should include('listing users')
|
||||
puts "Controller stack is functioning when rendering from a controller"
|
||||
|
||||
response.body.should include('hello miscellaneous')
|
||||
puts "All helper modules were included when rendering from a controller"
|
||||
end
|
||||
end
|
||||
"""
|
||||
Given a file named "spec/views/index.html.erb_spec.rb" with:
|
||||
"""
|
||||
require "spec_helper"
|
||||
describe "/users/index.html.erb" do
|
||||
|
||||
it "renders the view" do
|
||||
render
|
||||
rendered.should_not include('Original View')
|
||||
puts "Views are not being cached when rendering directly"
|
||||
|
||||
rendered.should include('listing users')
|
||||
puts "Controller stack is functioning when rendering directly"
|
||||
|
||||
rendered.should include('hello miscellaneous')
|
||||
puts "All helper modules were included when rendering directly"
|
||||
end
|
||||
end
|
||||
"""
|
||||
When I fire up a spork instance with "spork rspec"
|
||||
And the contents of "app/views/users/index.html.erb" are changed to:
|
||||
"""
|
||||
<%= reverse_text('listing users'.reverse) %>
|
||||
<%= misc_helper_method rescue nil %>
|
||||
<p>Here is a list of users</p>
|
||||
"""
|
||||
|
||||
And I run rspec --drb spec/controllers/users_controller_spec.rb
|
||||
Then the output should contain "Controller stack is functioning when rendering from a controller"
|
||||
And the output should contain "Views are not being cached when rendering from a controller"
|
||||
And the output should contain "All helper modules were included when rendering from a controller"
|
||||
|
||||
When I run rspec --drb spec/views/index.html.erb_spec.rb
|
||||
Then the output should contain "Controller stack is functioning when rendering directly"
|
||||
And the output should contain "Views are not being cached when rendering directly"
|
||||
And the output should contain "All helper modules were included when rendering directly"
|
||||
|
||||
Given the contents of "app/helpers/application_helper.rb" are changed to:
|
||||
"""
|
||||
module ApplicationHelper
|
||||
def make_it_loud(message)
|
||||
message.upcase
|
||||
end
|
||||
end
|
||||
"""
|
||||
And the contents of "app/views/users/index.html.erb" are changed to:
|
||||
"""
|
||||
<%= make_it_loud('listing users') %>
|
||||
"""
|
||||
And the contents of "spec/controllers/users_controller_spec.rb" are changed to:
|
||||
"""
|
||||
require "spec_helper"
|
||||
describe UsersController do
|
||||
render_views
|
||||
it "renders a page, using a method inherited from ApplicationController" do
|
||||
get :index
|
||||
response.body.should include('LISTING USERS')
|
||||
puts "Helpers aren't being cached"
|
||||
end
|
||||
end
|
||||
"""
|
||||
When I run rspec --drb spec/controllers/users_controller_spec.rb
|
||||
Then the output should contain "Helpers aren't being cached"
|
||||
|
|
@ -0,0 +1,92 @@
|
|||
Feature: Rails Integration
|
||||
To get a developer up and running quickly
|
||||
Spork automatically integrates with rails
|
||||
Providing default hooks and behaviors
|
||||
|
||||
Background: Rails App with RSpec and Spork
|
||||
Given I am in a fresh rails project named "test_rails_project"
|
||||
And a file named "spec/spec_helper.rb" with:
|
||||
"""
|
||||
require 'rubygems'
|
||||
require 'spork'
|
||||
|
||||
Spork.prefork do
|
||||
# Loading more in this block will cause your specs to run faster. However,
|
||||
# if you change any configuration or code from libraries loaded here, you'll
|
||||
# need to restart spork for it take effect.
|
||||
require File.dirname(__FILE__) + '/../config/environment.rb'
|
||||
require 'rspec'
|
||||
require 'rspec/rails'
|
||||
|
||||
#### this is for this test only #######
|
||||
$loaded_stuff << 'prefork block' ######
|
||||
#######################################
|
||||
end
|
||||
|
||||
Spork.each_run do
|
||||
# This code will be run each time you run your specs.
|
||||
|
||||
#### this is for this test only #######
|
||||
$loaded_stuff << 'each_run block' #####
|
||||
#######################################
|
||||
end
|
||||
"""
|
||||
And the application has a model, observer, route, and application helper
|
||||
Scenario: Analyzing files were preloaded
|
||||
When I run spork --diagnose
|
||||
Then the output should not contain "user_observer.rb"
|
||||
Then the output should not contain "user.rb"
|
||||
Then the output should not contain "app/controllers/application.rb"
|
||||
Then the output should not contain "app/controllers/application_controller.rb"
|
||||
Then the output should not contain "app/controllers/application_helper.rb"
|
||||
# Then the output should not contain "config/routes.rb"
|
||||
|
||||
Scenario: Running spork with a rails app and observers
|
||||
Given a file named "spec/did_it_work_spec.rb" with:
|
||||
"""
|
||||
require 'spec_helper'
|
||||
describe "Did it work?" do
|
||||
it "checks to see if all worked" do
|
||||
Spork.using_spork?.should == true
|
||||
(Rails.respond_to?(:logger) ? Rails.logger : ActionController::Base.logger).info "hey there"
|
||||
$loaded_stuff.should include('ActiveRecord::Base.establish_connection')
|
||||
$loaded_stuff.should include('User')
|
||||
$loaded_stuff.should include('UserObserver')
|
||||
$loaded_stuff.should include('ApplicationHelper')
|
||||
$loaded_stuff.should include('config/routes.rb')
|
||||
$loaded_stuff.should include('each_run block')
|
||||
$loaded_stuff.should include('prefork block')
|
||||
puts "Specs successfully run within spork, and all initialization files were loaded"
|
||||
end
|
||||
end
|
||||
"""
|
||||
When I fire up a spork instance with "spork rspec"
|
||||
And I run rspec --drb spec/did_it_work_spec.rb
|
||||
Then the error output should be empty
|
||||
And the output should contain "Specs successfully run within spork, and all initialization files were loaded"
|
||||
And the file "log/test.log" should include "hey there"
|
||||
|
||||
|
||||
Scenario: Running spork with a rails app and a non-standard port
|
||||
Given a file named "spec/did_it_work_spec.rb" with:
|
||||
"""
|
||||
describe "Did it work?" do
|
||||
it "checks to see if all worked" do
|
||||
Spork.using_spork?.should == true
|
||||
(Rails.respond_to?(:logger) ? Rails.logger : ActionController::Base.logger).info "hey there"
|
||||
$loaded_stuff.should include('ActiveRecord::Base.establish_connection')
|
||||
$loaded_stuff.should include('User')
|
||||
$loaded_stuff.should include('UserObserver')
|
||||
$loaded_stuff.should include('ApplicationHelper')
|
||||
$loaded_stuff.should include('config/routes.rb')
|
||||
$loaded_stuff.should include('each_run block')
|
||||
$loaded_stuff.should include('prefork block')
|
||||
puts "Specs successfully run within spork, and all initialization files were loaded"
|
||||
end
|
||||
end
|
||||
"""
|
||||
When I fire up a spork instance with "spork rspec --port 7000"
|
||||
And I run rspec --drb --drb-port 7000 spec/did_it_work_spec.rb
|
||||
Then the error output should be empty
|
||||
And the output should contain "Specs successfully run within spork, and all initialization files were loaded"
|
||||
And the file "log/test.log" should include "hey there"
|
||||
|
|
@ -0,0 +1,108 @@
|
|||
Feature: Spork Debugger integration
|
||||
As a developer
|
||||
I want to invoke the debugger my specs within Spork
|
||||
In order to drill in and figure out what's wrong
|
||||
|
||||
Scenario: Invoking the debugger via 'debugger'
|
||||
Given a file named "spec/spec_helper.rb" with:
|
||||
"""
|
||||
require 'rubygems'
|
||||
require 'spork'
|
||||
require 'spork/ext/ruby-debug'
|
||||
|
||||
Spork.prefork { require 'rspec' }
|
||||
Spork.each_run { }
|
||||
"""
|
||||
And a file named "spec/debugger_spec.rb" with:
|
||||
"""
|
||||
require 'spec_helper.rb'
|
||||
|
||||
describe "Debugger" do
|
||||
it "should debug" do
|
||||
2.times do |count|
|
||||
@message = "count = #{count}"
|
||||
debugger
|
||||
@message = nil
|
||||
end
|
||||
puts "it worked!"
|
||||
end
|
||||
end
|
||||
"""
|
||||
|
||||
When I fire up a spork instance with "spork rspec"
|
||||
And I run this in the background: rspec --drb spec/debugger_spec.rb
|
||||
|
||||
Then the spork window should output a line containing "Debug Session Started"
|
||||
|
||||
When I type this in the spork window: "e @message"
|
||||
Then the spork window should output a line containing "count = 0"
|
||||
|
||||
When I type this in the spork window: "continue"
|
||||
|
||||
When I type this in the spork window: "e @message"
|
||||
Then the spork window should output a line containing "count = 1"
|
||||
|
||||
When I type this in the spork window: "continue"
|
||||
|
||||
Then the spork window should output a line containing "Debug Session Terminated"
|
||||
And the output should contain "it worked!"
|
||||
|
||||
Scenario: When ruby-debug is already required and started.
|
||||
Given a file named "spec/spec_helper.rb" with:
|
||||
"""
|
||||
require 'rubygems'
|
||||
require 'spork'
|
||||
require 'ruby-debug'
|
||||
Debugger.start
|
||||
|
||||
require 'spork/ext/ruby-debug'
|
||||
|
||||
Spork.prefork { require 'rspec' }
|
||||
Spork.each_run { }
|
||||
"""
|
||||
|
||||
And a file named "spec/debugger_spec.rb" with:
|
||||
"""
|
||||
require File.dirname(__FILE__) + '/spec_helper.rb'
|
||||
|
||||
describe "Debugger" do
|
||||
it "should debug" do
|
||||
@message = "yup"
|
||||
debugger
|
||||
puts "it worked!"
|
||||
end
|
||||
end
|
||||
"""
|
||||
|
||||
When I fire up a spork instance with "spork rspec"
|
||||
And I run this in the background: rspec --drb spec/debugger_spec.rb
|
||||
|
||||
Then the spork window should output a line containing "Debug Session Started"
|
||||
|
||||
When I type this in the spork window: "e @message"
|
||||
Then the spork window should output a line containing "yup"
|
||||
|
||||
When I type this in the spork window: "continue"
|
||||
|
||||
Then the spork window should output a line containing "Debug Session Terminated"
|
||||
And the output should contain "it worked!"
|
||||
|
||||
Scenario: When ruby-debug is invoked during preload
|
||||
Given a file named "spec/spec_helper.rb" with:
|
||||
"""
|
||||
require 'rubygems'
|
||||
require 'spork'
|
||||
require 'spork/ext/ruby-debug'
|
||||
|
||||
STDERR.puts("Spork is ready and listening") # trick out the start spork step to believe spork is ready... naughty, but effective.
|
||||
@message = "it worked"
|
||||
debugger
|
||||
Spork.prefork { require 'rspec' }
|
||||
Spork.each_run { }
|
||||
"""
|
||||
|
||||
When I fire up a spork instance with "spork rspec"
|
||||
Then the spork window should output a line containing "spec_helper.rb"
|
||||
When I type this in the spork window: "e @message"
|
||||
Then the spork window should output a line containing "it worked"
|
||||
When I type this in the spork window: "continue"
|
||||
|
|
@ -0,0 +1,3 @@
|
|||
Given /^this scenario is pending.+/ do
|
||||
pending
|
||||
end
|
||||
|
|
@ -0,0 +1,67 @@
|
|||
Given /^I am in a fresh rails project named "(.+)"$/ do |folder_name|
|
||||
@current_dir = SporkWorld::SANDBOX_DIR
|
||||
# version_argument = ENV['RAILS_VERSION'] ? "_#{ENV['RAILS_VERSION']}_" : nil
|
||||
# run("#{SporkWorld::RUBY_BINARY} #{%x{which rails}.chomp} #{folder_name}")
|
||||
run(["rails", "new", folder_name].compact * " ")
|
||||
|
||||
if last_exit_status != 0
|
||||
puts "Couldn't generate project. Output:\nSTDERR:\n-------\n#{last_stderr}\n------\n\nSTDOUT:\n-------\n#{last_stdout}\n\n"
|
||||
last_exit_status.should == 0
|
||||
end
|
||||
@current_dir = File.join(File.join(SporkWorld::SANDBOX_DIR, folder_name))
|
||||
in_current_dir do
|
||||
FileUtils.ln_sf(ENV["BUNDLE_GEMFILE"], "Gemfile")
|
||||
FileUtils.ln_sf(ENV["BUNDLE_GEMFILE"] + ".lock", "Gemfile.lock")
|
||||
FileUtils.ln_sf(File.dirname(ENV["BUNDLE_GEMFILE"]) + "/.bundle", ".bundle")
|
||||
end
|
||||
end
|
||||
|
||||
|
||||
Given "the application has a model, observer, route, and application helper" do
|
||||
Given 'the following code appears in "config/application.rb" after /^end/:',
|
||||
"""
|
||||
ActiveRecord::Base.observers = [:user_observer]
|
||||
"""
|
||||
Given 'a file named "app/models/user.rb" with:',
|
||||
"""
|
||||
class User < ActiveRecord::Base
|
||||
$loaded_stuff << 'User'
|
||||
end
|
||||
"""
|
||||
Given 'a file named "app/models/user_observer.rb" with:',
|
||||
"""
|
||||
class UserObserver < ActiveRecord::Observer
|
||||
$loaded_stuff << 'UserObserver'
|
||||
end
|
||||
"""
|
||||
Given 'a file named "app/helpers/application_helper.rb" with:',
|
||||
"""
|
||||
module ApplicationHelper
|
||||
$loaded_stuff << 'ApplicationHelper'
|
||||
end
|
||||
"""
|
||||
Given 'the following code appears in "config/environment.rb" after /Rails::Initializer.run/:',
|
||||
"""
|
||||
config.active_record.observers = :user_observer
|
||||
"""
|
||||
Given 'the following code appears in "config/routes.rb" after /^end/:',
|
||||
"""
|
||||
$loaded_stuff << 'config/routes.rb'
|
||||
"""
|
||||
Given 'a file named "config/initializers/initialize_loaded_stuff.rb" with:',
|
||||
"""
|
||||
$loaded_stuff ||= []
|
||||
"""
|
||||
Given 'a file named "config/initializers/log_establish_connection_calls.rb" with:',
|
||||
"""
|
||||
class ActiveRecord::Base
|
||||
class << self
|
||||
def establish_connection_with_load_logging(*args)
|
||||
$loaded_stuff << 'ActiveRecord::Base.establish_connection'
|
||||
establish_connection_without_load_logging(*args)
|
||||
end
|
||||
alias_method_chain :establish_connection, :load_logging
|
||||
end
|
||||
end
|
||||
"""
|
||||
end
|
||||
|
|
@ -0,0 +1,115 @@
|
|||
Given /^I am in the directory "(.*)"$/ do |sandbox_dir_relative_path|
|
||||
path = File.join(SporkWorld::SANDBOX_DIR, sandbox_dir_relative_path)
|
||||
FileUtils.mkdir_p(path)
|
||||
@current_dir = File.join(path)
|
||||
end
|
||||
|
||||
Given /^a file named "([^\"]*)"$/ do |file_name|
|
||||
create_file(file_name, '')
|
||||
end
|
||||
|
||||
Given /^a file named "([^\"]*)" with:$/ do |file_name, file_content|
|
||||
create_file(file_name, file_content)
|
||||
end
|
||||
|
||||
When /^the contents of "([^\"]*)" are changed to:$/ do |file_name, file_content|
|
||||
create_file(file_name, file_content)
|
||||
end
|
||||
|
||||
# the following code appears in "config/environment.rb" after /Rails::Initializer.run/:
|
||||
Given /^the following code appears in "([^\"]*)" after \/([^\/]*)\/:$/ do |file_name, regex, content|
|
||||
regex = Regexp.new(regex)
|
||||
in_current_dir do
|
||||
content_lines = File.read(file_name).split("\n")
|
||||
0.upto(content_lines.length - 1) do |line_index|
|
||||
if regex.match(content_lines[line_index])
|
||||
content_lines.insert(line_index + 1, content)
|
||||
break
|
||||
end
|
||||
end
|
||||
File.open(file_name, 'wb') { |f| f << (content_lines * "\n") }
|
||||
end
|
||||
end
|
||||
|
||||
When /^I run (spork|rspec|cucumber)(| .*)$/ do |command, args|
|
||||
run("#{command} #{args}")
|
||||
end
|
||||
|
||||
When /^I run this in the background: (spork|rspec|cucumber)(| .*)$/ do |command, args|
|
||||
@background_script = run_in_background("#{command} #{args}")
|
||||
end
|
||||
|
||||
When /^I fire up a spork instance with "spork(.*)"$/ do |spork_opts|
|
||||
@spork_server = run_in_background("#{SporkWorld::RUBY_BINARY} -I #{Cucumber::LIBDIR} #{SporkWorld::BINARY} #{spork_opts}")
|
||||
|
||||
output = ""
|
||||
begin
|
||||
status = Timeout::timeout(15) do
|
||||
# Something that should be interrupted if it takes too much time...
|
||||
while line = @spork_server.stderr.gets
|
||||
output << line
|
||||
puts line
|
||||
break if line.include?("Spork is ready and listening")
|
||||
end
|
||||
end
|
||||
rescue Timeout::Error
|
||||
puts "I can't seem to launch Spork properly. Output was:\n#{output}"
|
||||
true.should == false
|
||||
end
|
||||
end
|
||||
|
||||
Then /^the spork window should output a line containing "(.+)"/ do |expected|
|
||||
output = ""
|
||||
begin
|
||||
status = Timeout::timeout(5) do
|
||||
# Something that should be interrupted if it takes too much time...
|
||||
while line = @spork_server.stdout.gets
|
||||
output << line
|
||||
puts line
|
||||
break if output.include?(expected)
|
||||
end
|
||||
end
|
||||
rescue Timeout::Error
|
||||
output.should include(expected)
|
||||
end
|
||||
end
|
||||
|
||||
When /^I type this in the spork window: "(.+)"/ do |line|
|
||||
@spork_server.stdin.puts(line)
|
||||
@spork_server.stdin.flush
|
||||
end
|
||||
|
||||
|
||||
Then /^the file "([^\"]*)" should include "([^\"]*)"$/ do |filename, content|
|
||||
in_current_dir do
|
||||
File.read(filename).should include(content)
|
||||
end
|
||||
end
|
||||
|
||||
Then /^the (error output|output) should contain$/ do |which, text|
|
||||
(which == "error output" ? last_stderr : last_stdout).should include(text)
|
||||
end
|
||||
|
||||
Then /^the (error output|output) should contain "(.+)"$/ do |which, text|
|
||||
(which == "error output" ? last_stderr : last_stdout).should include(text)
|
||||
end
|
||||
|
||||
Then /^the (error output|output) should match \/(.+)\/$/ do |which, regex|
|
||||
(which == "error output" ? last_stderr : last_stdout).should match(Regexp.new(regex))
|
||||
end
|
||||
|
||||
Then /^the (error output|output) should not contain$/ do |which, text|
|
||||
(which == "error output" ? last_stderr : last_stdout).should_not include(text)
|
||||
end
|
||||
|
||||
Then /^the (error output|output) should not contain "(.+)"$/ do |which, text|
|
||||
(which == "error output" ? last_stderr : last_stdout).should_not include(text)
|
||||
end
|
||||
|
||||
Then /^the (error output|output) should be empty$/ do |which|
|
||||
(which == "error output" ? last_stderr : last_stdout).should == ""
|
||||
end
|
||||
|
||||
Then /^the (error output|output) should be$/ do |which, text|
|
||||
(which == "error output" ? last_stderr : last_stdout).should == text
|
||||
end
|
||||
|
|
@ -0,0 +1,63 @@
|
|||
class BackgroundJob
|
||||
attr_reader :stdin, :stdout, :stderr, :pid
|
||||
def initialize(pid, stdin, stdout, stderr)
|
||||
@pid, @stdin, @stdout, @stderr = pid, stdin, stdout, stderr
|
||||
ObjectSpace.define_finalizer(self) { kill }
|
||||
end
|
||||
|
||||
def self.run(command)
|
||||
command = sanitize_params(command) if command.is_a?(Array)
|
||||
child_stdin, parent_stdin = IO::pipe
|
||||
parent_stdout, child_stdout = IO::pipe
|
||||
parent_stderr, child_stderr = IO::pipe
|
||||
|
||||
pid = Kernel.fork do
|
||||
[parent_stdin, parent_stdout, parent_stderr].each { |io| io.close }
|
||||
|
||||
STDIN.reopen(child_stdin)
|
||||
STDOUT.reopen(child_stdout)
|
||||
STDERR.reopen(child_stderr)
|
||||
|
||||
[child_stdin, child_stdout, child_stderr].each { |io| io.close }
|
||||
|
||||
exec command
|
||||
end
|
||||
|
||||
[child_stdin, child_stdout, child_stderr].each { |io| io.close }
|
||||
parent_stdin.sync = true
|
||||
|
||||
new(pid, parent_stdin, parent_stdout, parent_stderr)
|
||||
end
|
||||
|
||||
def self.sanitize_params(params)
|
||||
params.map { |p| p.gsub(' ', '\ ') }.join(" ")
|
||||
end
|
||||
|
||||
def kill(signal = 'TERM')
|
||||
if running?
|
||||
Process.kill(Signal.list[signal], @pid)
|
||||
true
|
||||
end
|
||||
end
|
||||
|
||||
def interrupt
|
||||
kill('INT')
|
||||
end
|
||||
|
||||
def running?
|
||||
return false unless @pid
|
||||
Process.getpgid(@pid)
|
||||
true
|
||||
rescue Errno::ESRCH
|
||||
false
|
||||
end
|
||||
|
||||
def wait(timeout = 1000)
|
||||
Timeout.timeout(timeout) do
|
||||
Process.wait(@pid)
|
||||
end
|
||||
true
|
||||
rescue Timeout::Error
|
||||
false
|
||||
end
|
||||
end
|
||||
|
|
@ -0,0 +1,41 @@
|
|||
require 'bundler'
|
||||
module BundlerHelpers
|
||||
extend self
|
||||
def install_bundle(dir)
|
||||
Dir.chdir(dir) do
|
||||
command = "env RUBYOPT= BUNDLE_GEMFILE=Gemfile bundle install"
|
||||
system(command)
|
||||
$?.exitstatus
|
||||
end
|
||||
end
|
||||
|
||||
def ensure_installed(dir)
|
||||
gemfile_lock = dir + "/Gemfile.lock"
|
||||
gemfile = dir + "/Gemfile"
|
||||
bundle_environment = dir + "/.bundle/environment.rb"
|
||||
case
|
||||
when File.exist?(gemfile_lock) && File.mtime(gemfile) > File.mtime(gemfile_lock)
|
||||
puts "Gemfile #{gemfile} has changed since it was locked. Re-locking..."
|
||||
FileUtils.rm(gemfile_lock)
|
||||
FileUtils.rm_rf(dir + "/.bundle")
|
||||
when ! File.exist?(bundle_environment)
|
||||
puts "Bundle #{gemfile} not installed. Installing..."
|
||||
when File.mtime(bundle_environment) < File.mtime(gemfile_lock)
|
||||
puts "#{gemfile_lock} is newer than #{bundle_environment}. Reinstalling"
|
||||
else
|
||||
return false
|
||||
end
|
||||
install_bundle(dir)
|
||||
end
|
||||
|
||||
def expand_gemfile(gemfile)
|
||||
possibilities = [File.expand_path(gemfile, Dir.pwd), SporkWorld::GEMFILES_ROOT + gemfile + "Gemfile"]
|
||||
possibilities.detect {|f| File.exist?(f)} || raise(RuntimeError, %(Gemfile not found:\n #{possibilities * "\n"}))
|
||||
end
|
||||
|
||||
def set_gemfile(gemfile)
|
||||
gemfile = expand_gemfile(gemfile || "rails3.0")
|
||||
ensure_installed(File.dirname(gemfile))
|
||||
ENV["BUNDLE_GEMFILE"] = gemfile.to_s
|
||||
end
|
||||
end
|
||||
|
|
@ -0,0 +1,105 @@
|
|||
require 'rubygems'
|
||||
require 'pathname'
|
||||
require 'fileutils'
|
||||
require 'forwardable'
|
||||
require 'tempfile'
|
||||
require 'rspec/expectations'
|
||||
require 'timeout'
|
||||
|
||||
require(File.dirname(__FILE__) + '/background_job.rb')
|
||||
|
||||
SPORK_ROOT = Pathname.new(File.expand_path('../../', File.dirname(__FILE__)))
|
||||
class SporkWorld
|
||||
RUBY_BINARY = File.join(Config::CONFIG['bindir'], Config::CONFIG['ruby_install_name'])
|
||||
BINARY = SPORK_ROOT + 'bin/spork'
|
||||
SANDBOX_DIR = SPORK_ROOT + "tmp/sandbox"
|
||||
GEMFILES_ROOT = SPORK_ROOT + "features/gemfiles"
|
||||
SPORK_LIBDIR = SPORK_ROOT + "lib"
|
||||
|
||||
extend Forwardable
|
||||
def_delegators SporkWorld, :sandbox_dir, :spork_lib_dir
|
||||
|
||||
def spork_lib_dir
|
||||
@spork_lib_dir ||= File.expand_path(File.join(File.dirname(__FILE__), '../../lib'))
|
||||
end
|
||||
|
||||
def initialize
|
||||
@current_dir = SANDBOX_DIR
|
||||
@background_jobs = []
|
||||
end
|
||||
|
||||
private
|
||||
attr_reader :last_exit_status, :last_stderr, :last_stdout, :background_jobs
|
||||
def last_stderr
|
||||
return @last_stderr if @last_stderr
|
||||
if @background_job
|
||||
@last_stderr = @background_job.stderr.read
|
||||
end
|
||||
end
|
||||
|
||||
|
||||
def last_stdout
|
||||
return @last_stdout if @last_stdout
|
||||
if @background_job
|
||||
@last_stdout = @background_job.stdout.read
|
||||
end
|
||||
end
|
||||
|
||||
def create_file(file_name, file_content)
|
||||
file_content.gsub!("SPORK_LIB", "'#{spork_lib_dir}'") # Some files, such as Rakefiles need to use the lib dir
|
||||
in_current_dir do
|
||||
FileUtils.mkdir_p(File.dirname(file_name))
|
||||
File.open(file_name, 'w') { |f| f << file_content }
|
||||
end
|
||||
end
|
||||
|
||||
def in_current_dir(&block)
|
||||
Dir.chdir(@current_dir, &block)
|
||||
end
|
||||
|
||||
def run(command)
|
||||
stderr_file = Tempfile.new('spork')
|
||||
stderr_file.close
|
||||
in_current_dir do
|
||||
@last_stdout = `env RUBYOPT= bundle exec #{command} 2> #{stderr_file.path}`
|
||||
@last_exit_status = $?.exitstatus
|
||||
end
|
||||
@last_stderr = IO.read(stderr_file.path)
|
||||
end
|
||||
|
||||
def run_in_background(command)
|
||||
in_current_dir do
|
||||
@background_job = BackgroundJob.run("env RUBYOPT= bundle exec " + command)
|
||||
end
|
||||
@background_jobs << @background_job
|
||||
@background_job
|
||||
end
|
||||
|
||||
def terminate_background_jobs
|
||||
if @background_jobs
|
||||
@background_jobs.each do |background_job|
|
||||
background_job.kill
|
||||
end
|
||||
end
|
||||
@background_jobs.clear
|
||||
@background_job = nil
|
||||
end
|
||||
end
|
||||
|
||||
require((SPORK_ROOT + "features/support/bundler_helpers.rb").to_s)
|
||||
BundlerHelpers.set_gemfile(ENV["GEMFILE"])
|
||||
|
||||
|
||||
World do
|
||||
SporkWorld.new
|
||||
end
|
||||
|
||||
Before do
|
||||
FileUtils.rm_rf SporkWorld::SANDBOX_DIR
|
||||
FileUtils.mkdir_p SporkWorld::SANDBOX_DIR
|
||||
end
|
||||
|
||||
After do
|
||||
# FileUtils.rm_rf SporkWorld::SANDBOX_DIR
|
||||
terminate_background_jobs
|
||||
end
|
||||
|
|
@ -0,0 +1,42 @@
|
|||
Feature: Unknown app frameworks
|
||||
To increase to usefulness of Spork
|
||||
Spork will work with unknown (or no) application frameworks
|
||||
|
||||
Scenario: Unsporked spec_helper
|
||||
|
||||
Given a file named "spec/spec_helper.rb" with:
|
||||
"""
|
||||
require 'rubygems'
|
||||
require 'spec'
|
||||
"""
|
||||
When I run spork
|
||||
Then the error output should contain "Using RSpec"
|
||||
Then the error output should match /You must bootstrap .+spec\/spec_helper\.rb to continue/
|
||||
|
||||
Scenario: Sporked spec_helper
|
||||
Given a file named "spec/spec_helper.rb" with:
|
||||
"""
|
||||
require 'rubygems'
|
||||
require 'spork'
|
||||
|
||||
Spork.prefork do
|
||||
require 'spec'
|
||||
end
|
||||
|
||||
Spork.each_run do
|
||||
$each_run
|
||||
end
|
||||
"""
|
||||
And a file named "spec/did_it_work_spec.rb" with:
|
||||
"""
|
||||
describe "Did it work?" do
|
||||
it "checks to see if all worked" do
|
||||
Spork.state.should == :using_spork
|
||||
puts "Specs successfully run within spork"
|
||||
end
|
||||
end
|
||||
"""
|
||||
When I fire up a spork instance with "spork rspec"
|
||||
And I run spec --drb spec/did_it_work_spec.rb
|
||||
Then the output should contain "Specs successfully run within spork"
|
||||
|
||||
|
|
@ -0,0 +1,155 @@
|
|||
$LOAD_PATH.unshift(File.expand_path(File.dirname(__FILE__))) unless $LOAD_PATH.include?(File.expand_path(File.dirname(__FILE__)))
|
||||
require 'pathname'
|
||||
module Spork
|
||||
BINARY = File.expand_path(File.dirname(__FILE__) + '/../bin/spork')
|
||||
LIBDIR = Pathname.new(File.expand_path(File.dirname(__FILE__)))
|
||||
|
||||
autoload :Server, (LIBDIR + 'spork/server').to_s
|
||||
autoload :TestFramework, (LIBDIR + 'spork/test_framework').to_s
|
||||
autoload :AppFramework, (LIBDIR + 'spork/app_framework').to_s
|
||||
autoload :RunStrategy, (LIBDIR + 'spork/run_strategy').to_s
|
||||
autoload :Runner, (LIBDIR + 'spork/runner').to_s
|
||||
autoload :Forker, (LIBDIR + 'spork/forker').to_s
|
||||
autoload :Diagnoser, (LIBDIR + 'spork/diagnoser').to_s
|
||||
autoload :GemHelpers, (LIBDIR + 'spork/gem_helpers').to_s
|
||||
|
||||
class << self
|
||||
# Run a block, during prefork mode. By default, if prefork is called twice in the same file and line number, the supplied block will only be ran once.
|
||||
#
|
||||
# == Parameters
|
||||
#
|
||||
# * +prevent_double_run+ - Pass false to disable double run prevention
|
||||
def prefork(prevent_double_run = true, &block)
|
||||
return if prevent_double_run && already_ran?(caller.first)
|
||||
yield
|
||||
end
|
||||
|
||||
# Run a block AFTER the fork occurs. By default, if prefork is called twice in the same file and line number, the supplied block will only be ran once.
|
||||
#
|
||||
# == Parameters
|
||||
#
|
||||
# * +prevent_double_run+ - Pass false to disable double run prevention
|
||||
def each_run(prevent_double_run = true, &block)
|
||||
return if prevent_double_run && already_ran?(caller.first)
|
||||
if state == :prefork
|
||||
each_run_procs << block
|
||||
else
|
||||
yield
|
||||
end
|
||||
end
|
||||
|
||||
# Run a block after specs are run.
|
||||
#
|
||||
# == Parameters
|
||||
#
|
||||
# * +prevent_double_run+ - Pass false to disable double run prevention
|
||||
def after_each_run(prevent_double_run = true, &block)
|
||||
return if prevent_double_run && already_ran?(caller.first)
|
||||
after_each_run_procs << block
|
||||
end
|
||||
|
||||
def using_spork?
|
||||
state != :not_using_spork
|
||||
end
|
||||
|
||||
def state
|
||||
@state ||= :not_using_spork
|
||||
end
|
||||
|
||||
# Used by the server. Called when loading the prefork blocks of the code.
|
||||
def exec_prefork(&block)
|
||||
@state = :prefork
|
||||
yield
|
||||
end
|
||||
|
||||
# Used by the server. Called to run all of the prefork blocks.
|
||||
def exec_each_run(&block)
|
||||
@state = :run
|
||||
activate_after_each_run_at_exit_hook
|
||||
each_run_procs.each { |p| p.call }
|
||||
each_run_procs.clear
|
||||
yield if block_given?
|
||||
end
|
||||
|
||||
# Used by the server. Called to run all of the after_each_run blocks.
|
||||
def exec_after_each_run
|
||||
# processes in reverse order similar to at_exit
|
||||
while p = after_each_run_procs.pop; p.call; end
|
||||
true
|
||||
end
|
||||
|
||||
# Traps an instance method of a class (or module) so any calls to it don't actually run until Spork.exec_each_run
|
||||
def trap_method(klass, method_name)
|
||||
method_name_without_spork, method_name_with_spork = alias_method_names(method_name, :spork)
|
||||
|
||||
klass.class_eval <<-EOF, __FILE__, __LINE__ + 1
|
||||
alias :#{method_name_without_spork} :#{method_name} unless method_defined?(:#{method_name_without_spork})
|
||||
def #{method_name}(*args, &block)
|
||||
Spork.each_run(false) do
|
||||
#{method_name_without_spork}(*args, &block)
|
||||
end
|
||||
end
|
||||
EOF
|
||||
end
|
||||
|
||||
# Same as trap_method, but for class methods instead
|
||||
def trap_class_method(klass, method_name)
|
||||
trap_method((class << klass; self; end), method_name)
|
||||
end
|
||||
|
||||
def detect_and_require(subfolder)
|
||||
([LIBDIR.to_s] + other_spork_gem_load_paths).uniq.each do |gem_path|
|
||||
Dir.glob(File.join(gem_path, subfolder)).each { |file| require file }
|
||||
end
|
||||
end
|
||||
|
||||
# This method is used to auto-discover peer plugins such as spork-testunit.
|
||||
def other_spork_gem_load_paths
|
||||
@other_spork_gem_load_paths ||= Spork::GemHelpers.latest_load_paths.grep(/spork/).select do |g|
|
||||
not g.match(%r{/spork-[0-9\-.]+/lib}) # don't include other versions of spork
|
||||
end
|
||||
end
|
||||
|
||||
private
|
||||
def activate_after_each_run_at_exit_hook
|
||||
Kernel.module_eval do
|
||||
def at_exit(&block)
|
||||
Spork.after_each_run(false, &block)
|
||||
end
|
||||
end
|
||||
end
|
||||
|
||||
def alias_method_names(method_name, feature)
|
||||
/^(.+?)([\?\!]{0,1})$/.match(method_name.to_s)
|
||||
["#{$1}_without_spork#{$2}", "#{$1}_with_spork#{$2}"]
|
||||
end
|
||||
|
||||
def already_ran
|
||||
@already_ran ||= []
|
||||
end
|
||||
|
||||
def expanded_caller(caller_line)
|
||||
file, line = caller_line.split(/:(\d+)/)
|
||||
line.gsub(/:.+/, '')
|
||||
expanded = File.expand_path(file, Dir.pwd) + ":" + line
|
||||
if ENV['OS'] == 'Windows_NT' # windows
|
||||
expanded = expanded[2..-1]
|
||||
end
|
||||
expanded
|
||||
end
|
||||
|
||||
def already_ran?(caller_script_and_line)
|
||||
return true if already_ran.include?(expanded_caller(caller_script_and_line))
|
||||
already_ran << expanded_caller(caller_script_and_line)
|
||||
false
|
||||
end
|
||||
|
||||
def each_run_procs
|
||||
@each_run_procs ||= []
|
||||
end
|
||||
|
||||
def after_each_run_procs
|
||||
@after_each_run_procs ||= []
|
||||
end
|
||||
end
|
||||
end
|
||||
|
|
@ -0,0 +1,80 @@
|
|||
class Spork::AppFramework
|
||||
# A hash of procs where the key is the class name, and the proc takes no arguments and returns true if it detects that said application framework is being used in the project.
|
||||
#
|
||||
# The key :Rails maps to Spork::AppFramework::Rails
|
||||
#
|
||||
# This is used to reduce the amount of code needed to be loaded - only the detected application framework's support code is loaded.
|
||||
SUPPORTED_FRAMEWORKS = {
|
||||
:Padrino => lambda {
|
||||
File.exist?("config/boot.rb") && File.read("config/boot.rb").include?('PADRINO')
|
||||
},
|
||||
:Rails => lambda {
|
||||
File.exist?("config/environment.rb") && (
|
||||
File.read("config/environment.rb").include?('RAILS_GEM_VERSION') ||
|
||||
(File.exist?("config/application.rb") && File.read("config/application.rb").include?("Rails::Application"))
|
||||
)
|
||||
}
|
||||
} unless defined? SUPPORTED_FRAMEWORKS
|
||||
|
||||
def self.setup_autoload
|
||||
([:Unknown] + SUPPORTED_FRAMEWORKS.keys).each do |name|
|
||||
autoload name, File.join(File.dirname(__FILE__), "app_framework", name.to_s.downcase)
|
||||
end
|
||||
end
|
||||
|
||||
# Iterates through all SUPPORTED_FRAMEWORKS and returns the symbolic name of the project application framework detected. Otherwise, returns :Unknown
|
||||
def self.detect_framework_name
|
||||
SUPPORTED_FRAMEWORKS.each do |key, value|
|
||||
return key if value.call
|
||||
end
|
||||
:Unknown
|
||||
end
|
||||
|
||||
# Same as detect_framework_name, but returns an instance of the specific AppFramework class.
|
||||
def self.detect_framework
|
||||
name = detect_framework_name
|
||||
self[name]
|
||||
end
|
||||
|
||||
# Initializes, stores, and returns a singleton instance of the named AppFramework.
|
||||
#
|
||||
# == Parameters
|
||||
#
|
||||
# # +name+ - A symbolic name of a AppFramework subclass
|
||||
#
|
||||
# == Example
|
||||
#
|
||||
# Spork::AppFramework[:Rails]
|
||||
def self.[](name)
|
||||
instances[name] ||= const_get(name).new
|
||||
end
|
||||
|
||||
def self.short_name
|
||||
name.gsub('Spork::AppFramework::', '')
|
||||
end
|
||||
|
||||
# If there is some stuff out of the box that the Spork can do to speed up tests without the test helper file being bootstrapped, this should return false.
|
||||
def bootstrap_required?
|
||||
entry_point.nil?
|
||||
end
|
||||
|
||||
# Abstract: The path to the file that loads the project environment, ie config/environment.rb. Returns nil if there is none.
|
||||
def entry_point
|
||||
raise NotImplementedError
|
||||
end
|
||||
|
||||
def preload(&block)
|
||||
yield
|
||||
end
|
||||
|
||||
def short_name
|
||||
self.class.short_name
|
||||
end
|
||||
|
||||
protected
|
||||
def self.instances
|
||||
@instances ||= {}
|
||||
end
|
||||
end
|
||||
|
||||
Spork::AppFramework.setup_autoload
|
||||
|
|
@ -0,0 +1,22 @@
|
|||
class Spork::AppFramework::Padrino < Spork::AppFramework
|
||||
|
||||
def preload(&block)
|
||||
STDERR.puts "Preloading Padrino environment"
|
||||
STDERR.flush
|
||||
ENV["PADRINO_ENV"] ||= "test"
|
||||
require boot_file
|
||||
# Make it so that we don't have to restart Spork if we change, say, a model or routes
|
||||
Spork.each_run { ::Padrino.reload! }
|
||||
yield
|
||||
end
|
||||
|
||||
def entry_point
|
||||
@entry_point ||= File.expand_path("config/boot.rb", Dir.pwd)
|
||||
end
|
||||
alias :boot_file :entry_point
|
||||
|
||||
def boot_contents
|
||||
@boot_contents ||= File.read(boot_file)
|
||||
end
|
||||
|
||||
end
|
||||
|
|
@ -0,0 +1,82 @@
|
|||
class Spork::AppFramework::Rails < Spork::AppFramework
|
||||
|
||||
def preload(&block)
|
||||
STDERR.puts "Preloading Rails environment"
|
||||
STDERR.flush
|
||||
ENV["RAILS_ENV"] ||= 'test'
|
||||
preload_rails
|
||||
yield
|
||||
end
|
||||
|
||||
def entry_point
|
||||
@entry_point ||= File.expand_path("config/environment.rb", Dir.pwd)
|
||||
end
|
||||
|
||||
alias :environment_file :entry_point
|
||||
|
||||
def boot_file
|
||||
@boot_file ||= File.join(File.dirname(environment_file), 'boot')
|
||||
end
|
||||
|
||||
def application_file
|
||||
@application_file ||= File.join(File.dirname(environment_file), 'application')
|
||||
end
|
||||
|
||||
def environment_contents
|
||||
@environment_contents ||= File.read(environment_file)
|
||||
end
|
||||
|
||||
def vendor
|
||||
@vendor ||= File.expand_path("vendor/rails", Dir.pwd)
|
||||
end
|
||||
|
||||
def deprecated_version
|
||||
@version ||= (
|
||||
if /^[^#]*RAILS_GEM_VERSION\s*=\s*["']([!~<>=]*\s*[\d.]+)["']/.match(environment_contents)
|
||||
$1
|
||||
else
|
||||
nil
|
||||
end
|
||||
)
|
||||
end
|
||||
|
||||
def preload_rails
|
||||
if deprecated_version && (not /^3/.match(deprecated_version))
|
||||
puts "This version of spork only supports Rails 3.0. To use spork with rails 2.3.x, downgrade to spork 0.8.x."
|
||||
exit 1
|
||||
end
|
||||
require application_file
|
||||
::Rails.application
|
||||
::Rails::Engine.class_eval do
|
||||
def eager_load!
|
||||
# turn off eager_loading, all together
|
||||
end
|
||||
end
|
||||
# Spork.trap_method(::AbstractController::Helpers::ClassMethods, :helper)
|
||||
Spork.trap_method(::ActiveModel::Observing::ClassMethods, :instantiate_observers)
|
||||
Spork.each_run { ActiveRecord::Base.establish_connection rescue nil } if Object.const_defined?(:ActiveRecord)
|
||||
|
||||
|
||||
AbstractController::Helpers::ClassMethods.module_eval do
|
||||
def helper(*args, &block)
|
||||
([args].flatten - [:all]).each do |arg|
|
||||
next unless arg.is_a?(String)
|
||||
filename = arg + "_helper"
|
||||
unless ::ActiveSupport::Dependencies.search_for_file(filename)
|
||||
# this error message must raise in the format such that LoadError#path returns the filename
|
||||
raise LoadError.new("Missing helper file helpers/%s.rb" % filename)
|
||||
end
|
||||
end
|
||||
|
||||
Spork.each_run(false) do
|
||||
modules_for_helpers(args).each do |mod|
|
||||
add_template_helper(mod)
|
||||
end
|
||||
|
||||
_helpers.module_eval(&block) if block_given?
|
||||
end
|
||||
end
|
||||
end
|
||||
end
|
||||
|
||||
end
|
||||
|
|
@ -0,0 +1,6 @@
|
|||
# This is used if no supported appliction framework is detected
|
||||
class Spork::AppFramework::Unknown < Spork::AppFramework
|
||||
def entry_point
|
||||
nil
|
||||
end
|
||||
end
|
||||
|
|
@ -0,0 +1,25 @@
|
|||
# This class is mainly used for testing.
|
||||
# When included (and used), it gives us an opportunity to stub out the output streams used for a given class
|
||||
module Spork::CustomIOStreams
|
||||
def self.included(klass)
|
||||
klass.send(:extend, ::Spork::CustomIOStreams::ClassMethods)
|
||||
end
|
||||
|
||||
def stderr
|
||||
self.class.stderr
|
||||
end
|
||||
|
||||
def stdout
|
||||
self.class.stdout
|
||||
end
|
||||
|
||||
module ClassMethods
|
||||
def stderr
|
||||
$stderr
|
||||
end
|
||||
|
||||
def stdout
|
||||
$stdout
|
||||
end
|
||||
end
|
||||
end
|
||||
|
|
@ -0,0 +1,105 @@
|
|||
# The Diagnoser hooks into load and require and keeps track of when files are required / loaded, and who loaded them.
|
||||
# It's used when you run spork --diagnose
|
||||
#
|
||||
# = Example
|
||||
#
|
||||
# Spork::Diagnoser.install_hook!('/path/env.rb', '/path')
|
||||
# require '/path/to/env.rb'
|
||||
# Spork::Diagnoser.output_results(STDOUT)
|
||||
class Spork::Diagnoser
|
||||
class << self
|
||||
def loaded_files
|
||||
@loaded_files ||= {}
|
||||
end
|
||||
|
||||
# Installs the diagnoser hook into Kernel#require and Kernel#load
|
||||
#
|
||||
# == Parameters
|
||||
#
|
||||
# * +entry_file+ - The file that is used to load the project. Used to filter the backtrace so anything that happens after it is hidden.
|
||||
# * +dir+ - The project directory. Any file loaded outside of this directory will not be logged.
|
||||
def install_hook!(entry_file = nil, dir = Dir.pwd)
|
||||
@dir = File.expand_path(Dir.pwd, dir)
|
||||
@entry_file = entry_file
|
||||
|
||||
Kernel.class_eval do
|
||||
alias :require_without_diagnoser :require
|
||||
alias :load_without_diagnoser :load
|
||||
|
||||
def require(string)
|
||||
::Spork::Diagnoser.add_included_file(string, caller)
|
||||
require_without_diagnoser(string)
|
||||
end
|
||||
private :require
|
||||
|
||||
def load(string, wrap = false)
|
||||
::Spork::Diagnoser.add_included_file(string, caller)
|
||||
load_without_diagnoser(string)
|
||||
end
|
||||
private :load
|
||||
end
|
||||
end
|
||||
|
||||
def add_included_file(filename, callstack)
|
||||
filename = expand_filename(filename)
|
||||
return unless File.exist?(filename)
|
||||
loaded_files[filename] = filter_callstack(caller) if subdirectory?(filename)
|
||||
end
|
||||
|
||||
# Uninstall the hook. Generally useful only for testing the Diagnoser.
|
||||
def remove_hook!
|
||||
return unless Kernel.private_instance_methods.map(&:to_sym).include?(:require_without_diagnoser)
|
||||
Kernel.class_eval do
|
||||
alias :require :require_without_diagnoser
|
||||
alias :load :load_without_diagnoser
|
||||
|
||||
undef_method(:require_without_diagnoser)
|
||||
undef_method(:load_without_diagnoser)
|
||||
end
|
||||
true
|
||||
end
|
||||
|
||||
# output the results of a diagnostic run.
|
||||
#
|
||||
# == Parameters
|
||||
#
|
||||
# * +stdout+ - An IO stream to output the results to.
|
||||
def output_results(stdout)
|
||||
project_prefix = Dir.pwd + "/"
|
||||
minimify = lambda { |f| f.gsub(project_prefix, '')}
|
||||
stdout.puts "- Spork Diagnosis -\n"
|
||||
stdout.puts "-- Summary --"
|
||||
stdout.puts loaded_files.keys.sort.map(&minimify)
|
||||
stdout.puts "\n\n\n"
|
||||
stdout.puts "-- Detail --"
|
||||
loaded_files.keys.sort.each do |file|
|
||||
stdout.puts "\n\n\n--- #{minimify.call(file)} ---\n"
|
||||
stdout.puts loaded_files[file].map(&minimify)
|
||||
end
|
||||
end
|
||||
|
||||
private
|
||||
def filter_callstack(callstack, entry_file = @entry_file)
|
||||
callstack.pop until callstack.empty? || callstack.last.include?(@entry_file) if @entry_file
|
||||
callstack.map do |line|
|
||||
next if line.include?('lib/spork/diagnoser.rb')
|
||||
line.gsub!('require_without_diagnoser', 'require')
|
||||
line
|
||||
end.compact
|
||||
end
|
||||
|
||||
def expand_filename(filename)
|
||||
([Dir.pwd] + $:).each do |attempted_path|
|
||||
attempted_filename = File.expand_path(filename, attempted_path)
|
||||
return attempted_filename if File.file?(attempted_filename)
|
||||
attempted_filename = attempted_filename + ".rb"
|
||||
return attempted_filename if File.file?(attempted_filename)
|
||||
end
|
||||
filename
|
||||
end
|
||||
|
||||
def subdirectory?(directory)
|
||||
File.expand_path(directory, Dir.pwd).include?(@dir)
|
||||
end
|
||||
end
|
||||
end
|
||||
|
|
@ -0,0 +1,14 @@
|
|||
Spork.each_run do
|
||||
::ActiveSupport.const_defined?(:Dependencies) ?
|
||||
::ActiveSupport::Dependencies.mechanism = :load :
|
||||
::Dependencies.mechanism = :load
|
||||
|
||||
require 'action_controller/dispatcher'
|
||||
dispatcher = ::ActionController::Dispatcher.new($stdout)
|
||||
|
||||
if ::ActionController::Dispatcher.respond_to?(:reload_application)
|
||||
::ActionController::Dispatcher.reload_application
|
||||
else
|
||||
dispatcher.reload_application
|
||||
end
|
||||
end if Spork.using_spork?
|
||||
|
|
@ -0,0 +1,150 @@
|
|||
require 'socket'
|
||||
require 'forwardable'
|
||||
|
||||
begin
|
||||
require 'ruby-debug'
|
||||
|
||||
# Experimental!
|
||||
|
||||
class SporkDebugger
|
||||
DEFAULT_PORT = 10_123
|
||||
HOST = '127.0.0.1'
|
||||
|
||||
extend Forwardable
|
||||
def_delegators :state, *[:prepare_debugger, :initialize]
|
||||
attr_reader :state
|
||||
|
||||
class << self
|
||||
attr_reader :instance
|
||||
def run
|
||||
@instance ||= new
|
||||
end
|
||||
end
|
||||
|
||||
def initialize
|
||||
@state = SporkDebugger::PreloadState.new
|
||||
Spork.send(:each_run_procs).unshift(lambda do
|
||||
@state = @state.transition_to_each_run_state
|
||||
end)
|
||||
end
|
||||
|
||||
module NetworkHelpers
|
||||
def find_port(starting_with)
|
||||
port = starting_with
|
||||
begin
|
||||
server = TCPServer.new(HOST, port)
|
||||
server.close
|
||||
rescue Errno::EADDRINUSE
|
||||
port += 1
|
||||
retry
|
||||
end
|
||||
|
||||
port
|
||||
end
|
||||
end
|
||||
|
||||
class PreloadState
|
||||
include NetworkHelpers
|
||||
def initialize
|
||||
Spork.each_run { install_hook }
|
||||
listen_for_connection_signals
|
||||
end
|
||||
|
||||
def finish
|
||||
@tcp_service.close; @tcp_service = nil;
|
||||
end
|
||||
|
||||
def transition_to_each_run_state
|
||||
finish
|
||||
SporkDebugger::EachRunState.new(@port)
|
||||
end
|
||||
|
||||
protected
|
||||
def install_hook
|
||||
Kernel.class_eval do
|
||||
alias :debugger_without_spork_hook :debugger
|
||||
def debugger(steps = 1)
|
||||
SporkDebugger.instance.prepare_debugger
|
||||
debugger_without_spork_hook
|
||||
end
|
||||
end
|
||||
end
|
||||
|
||||
def listen_for_connection_signals
|
||||
@port = SporkDebugger::DEFAULT_PORT
|
||||
begin
|
||||
@tcp_service = TCPServer.new(SporkDebugger::HOST, @port)
|
||||
rescue Errno::EADDRINUSE
|
||||
@port += 1
|
||||
retry
|
||||
end
|
||||
Thread.new { main_loop }
|
||||
end
|
||||
|
||||
def main_loop
|
||||
while @tcp_service do
|
||||
socket = @tcp_service.accept
|
||||
port = Marshal.load(socket)
|
||||
Marshal.dump(true, socket)
|
||||
connect_rdebug_client(port)
|
||||
socket.close
|
||||
end
|
||||
rescue => e
|
||||
puts "error: #{e.class} - #{e}"
|
||||
end
|
||||
|
||||
def connect_rdebug_client(port = Debugger::PORT)
|
||||
puts "\n\n - Debug Session Started - \n\n\n"
|
||||
begin
|
||||
Debugger.start_client(SporkDebugger::HOST, port)
|
||||
rescue Errno::EPIPE, Errno::ECONNRESET, Errno::ECONNREFUSED
|
||||
end
|
||||
puts "\n\n - Debug Session Terminated - \n\n\n"
|
||||
end
|
||||
end
|
||||
|
||||
class EachRunState
|
||||
include NetworkHelpers
|
||||
def initialize(connection_request_port)
|
||||
@connection_request_port = connection_request_port
|
||||
end
|
||||
|
||||
def prepare_debugger
|
||||
return if @debugger_prepared
|
||||
@debugger_prepared = true
|
||||
port, cport = start_rdebug_server
|
||||
signal_spork_server_to_connect_to_rdebug_server(port)
|
||||
wait_for_connection
|
||||
puts "\n\n - breakpoint - see your spork server for the debug terminal - \n\n"
|
||||
end
|
||||
|
||||
def signal_spork_server_to_connect_to_rdebug_server(rdebug_server_port)
|
||||
socket = TCPSocket.new(SporkDebugger::HOST, @connection_request_port)
|
||||
Marshal.dump(rdebug_server_port, socket)
|
||||
response = Marshal.load(socket)
|
||||
socket.close
|
||||
end
|
||||
|
||||
def start_rdebug_server
|
||||
Debugger.stop if Debugger.started?
|
||||
port = find_port(Debugger::PORT)
|
||||
cport = find_port(port + 1)
|
||||
Debugger.start_remote(SporkDebugger::HOST, [port, cport]) do
|
||||
Debugger.run_init_script(StringIO.new)
|
||||
end
|
||||
Debugger.start
|
||||
[port, cport]
|
||||
end
|
||||
|
||||
protected
|
||||
def wait_for_connection
|
||||
while Debugger.handler.interface.nil?; sleep 0.10; end
|
||||
end
|
||||
end
|
||||
end
|
||||
|
||||
Spork.prefork { SporkDebugger.run } if Spork.using_spork?
|
||||
|
||||
rescue LoadError
|
||||
raise LoadError, "Your project has loaded spork/ext/ruby-debug, which relies on the ruby-debug gem. It appears that ruby-debug is not installed. Please install it."
|
||||
end
|
||||
|
|
@ -0,0 +1,71 @@
|
|||
# A helper class that allows you to run a block inside of a fork, and then get the result from that block.
|
||||
#
|
||||
# == Example:
|
||||
#
|
||||
# forker = Spork::Forker.new do
|
||||
# sleep 3
|
||||
# "success"
|
||||
# end
|
||||
#
|
||||
# forker.result # => "success"
|
||||
class Spork::Forker
|
||||
|
||||
# Raised if the fork died (was killed) before it sent it's response back.
|
||||
class ForkDiedException < Exception; end
|
||||
def initialize(&block)
|
||||
return unless block_given?
|
||||
@child_io, @server_io = UNIXSocket.socketpair
|
||||
@child_pid = Kernel.fork do
|
||||
begin
|
||||
@server_io.close
|
||||
Marshal.dump(yield, @child_io)
|
||||
# wait for the parent to acknowledge receipt of the result.
|
||||
master_response = Marshal.load(@child_io)
|
||||
rescue EOFError
|
||||
nil
|
||||
rescue Exception => e
|
||||
puts "Exception encountered: #{e.inspect}\nbacktrace:\n#{e.backtrace * %(\n)}"
|
||||
end
|
||||
|
||||
# terminate, skipping any at_exit blocks.
|
||||
exit!(0)
|
||||
end
|
||||
@child_io.close
|
||||
end
|
||||
|
||||
# Wait for the fork to finish running, and then return its return value.
|
||||
#
|
||||
# If the fork was aborted, then result returns nil.
|
||||
def result
|
||||
return unless running?
|
||||
result_thread = Thread.new do
|
||||
begin
|
||||
@result = Marshal.load(@server_io)
|
||||
Marshal.dump('ACK', @server_io)
|
||||
rescue ForkDiedException, EOFError
|
||||
@result = nil
|
||||
end
|
||||
end
|
||||
Process.wait(@child_pid)
|
||||
result_thread.raise(ForkDiedException) if @result.nil?
|
||||
@child_pid = nil
|
||||
@result
|
||||
end
|
||||
|
||||
# abort the current running fork
|
||||
def abort
|
||||
if running?
|
||||
Process.kill(Signal.list['TERM'], @child_pid)
|
||||
@child_pid = nil
|
||||
true
|
||||
end
|
||||
end
|
||||
|
||||
def running?
|
||||
return false unless @child_pid
|
||||
Process.getpgid(@child_pid)
|
||||
true
|
||||
rescue Errno::ESRCH
|
||||
false
|
||||
end
|
||||
end
|
||||
Some files were not shown because too many files have changed in this diff Show More
Loading…
Reference in New Issue