dec 31, 2015 · 2 min

Using Travis CI to Validate Awesome Lists

I maintain several awesome-style lists, namely: awesome-react-native, awesome-devenv, awesome-colorschemes, awesome-weekly, and awesome-beginners.

Some more complete than others, and being an OCD curator, some really serve my own needs.

What I (and probably others) didn’t realize, is that since the web is constantly changing, these lists are subject to breaking — directly proportional to the number of links they host.

Travis CI

I’ve been using Travis CI for nearly every serious project I host on Github, and it is really valuable, especially when I have an open-source project which is really nothing more than an idea, execution, and good intentions of the original author and community.

For my awesome-lists, to make sure they don’t break without my noticing, and to make sure each pull request contains a valid link, I thought of using Travis CI along with a custom script to parse and verify links from a markdown document.

Surely this is not really testing code, and I hope it still is blessed by Travis CI (thanks for this awesome free service for open-source projects, guys!) but still it can be looked at as a sort of an integration test.

The validation script is pretty naive, at the expense of being dead simple and yet effective:

BASE_URI = ENV['BASE_URI'] || 'https://github.com/jondot/awesome-react-native'

doc = Nokogiri::HTML(Kramdown::Document.new(open('README.md').read).to_html)
links = doc.css('a').to_a
puts "Validating #{links.count} links..."

invalids = []
Parallel.each(links, :in_threads => 4) do |link|
  begin
    uri = URI.join(BASE_URI, link.attr('href'))
    open(uri)
    putc('.')
  rescue
    putc('F')
    invalids << link
  end
end

unless invalids.empty?
  puts "\n\nFailed links:"
  invalids.each do |link|
    puts "- #{link.text}"
  end
  puts "Done with errors."
  exit(1)
end

puts "\nDone."

And the Travis setup should be something like this:

language: ruby
rvm:
- 2.2
script: bundle exec ruby validate.rb

And then, on each commit a build runs. A failing build looks like this:

Validating 254 links...
.........................................................F....................................................................................................................................................................................................
Failed links:
- react-native-context-menu
Done with errors.

And is nicely exposed with a Travis badge on my READMEs.

Check this out live on my awesome-react-native list.

← all posts