# Starting from scratch? # You will need 'git' installed. If using a Mac, probably best # installed via MacPorts' 'port' command, which you may need # to install as well. You may also need to install qt4 or qt4-mac. # Installation of these binaries, ruby and rubygems is beyond # the scope of this README file. git clone [email protected]:ccls/odms.git # Or just give it a pull to update. git pull # create a big hash (rake secret) and insert as your secret token cp config/initializers/secret_token.rb.example config/initializers/secret_token.rb # define your databases cp config/database.yml.example config/database.yml # you may have to copy in some mail credentials ... cp config/initializers/mail.rb.example config/initializers/mail.rb # Assuming that ruby 1.9.3 is installed which now includes rubygems. # Should probably update to the latest rubygems (1.8.24 as of 20120619) # (Not really sure if this is actually necessary in 1.9.3, # but I did it and everything still seems to work.) gem update --system # Install bundler which will be managing all of the gems gem install bundler # Install all of the gems bundle install # drop existing dev db. # This is obviously destructive so don't do it # unless you know that. mysqladmin -u root drop odms_development # create new empty dev db mysqladmin -u root create odms_development # create all the other databases (could skip the above create) rake db:create:all # Run all of the migrations. rake db:migrate # Perhaps prep the test db rake db:test:prepare # The test fixtures contain REAL data to be used in # the REAL application and so it MUST be installed. # Install most test fixtures in database. # This is destructive and will remove all existing # before adding those in the fixtures. rake app:update # OR Install zip_code and county csv fixtures in database (takes a while) # ( this will do app:update as well ) # As the zip_code and county info doesn't really ever change, # I separated it out. As Rails 3 doesn't do csv fixtures, # I had to custom write the task as well. rake app:full_update # Add expected users and roles (must be after roles are loaded) # This isn't destructive, so you can call this as many times as you like. rake app:users:add_default # Install "production" pages and guides if necessary (as of Mar 25, 2012) # This is also destructive. Once in production, you may never # want to do this again as the pages and guides can be maintained # through the application. #rake db:fixtures:load FIXTURES_PATH=production/fixtures/ FIXTURES=pages,guides # For the moment, we are trying sunspot and solr for subject indexing. # All seems to be working well, but sunspot needs to be started # before the app. This may require RAILS_ENV=yourenvironment # (test,development,production). bundle exec rake sunspot:solr:start # start the server (common development mode) script/rails server # # That will use the development environment by default unless # the environment variable RAILS_ENV is set to something # or the option -e or --environment is given. # # In addition, this will default to port 3000 unless # the -p or --port option is given. # # This will also run the server in non-Daemon mode # unless the -d or --daemon option is given. # Daemon mode effectively starts the server in the background. # # This will use Webrick by default. I don't know if it # make much difference, but you can also specify mongrel. # # OR start the server on our production server # (RAILS_ENV is set in .tcshrc so not NEEDED.) bundle exec rake sunspot:solr:start RAILS_ENV=production script/rails server -p 3000 -e production -d mongrel # You should also be able to run script/rails console # As well as ... autotest
# Sunspot should be started first bundle exec rake sunspot:solr:start RAILS_ENV=test # One pass, running all tests ... rake test # Or, since this is the default task, just ... rake # One pass, running all tests and checking code coverage ... rake test:coverage # DO NOT edit the app while this is running, or the resulting # coverage information will be corrupt. # Continual testing triggered by file changes ... autotest # Be aware that changes don't necessarily trigger # all necessary testing of the given change. # They just trigger the associated mapped files.
-
imported ucb_ldap so could modify and remove incompatible ruby-net-ldap requirement
-
FasterCSV replaced with CSV
-
must manually “require ‘csv’” instead though
-
-
no longer use rubygems binary as is include with ruby
-
paperclip dropped ‘to_file’ method
-
using queued_for_write instead
-
-
yaml/encoding no longer exists (was using in sorting exported fixtures)
-
apparently functionality is included
-
-
Factories explicitly need “sequence” for my methods
-
assert_select requires more explicit hash use
-
doesn’t parse american date so need american_date gem
-
installed american_date gem which fixed a lot of unexpected problems
-
-
rcov no longer compatible with suggestion of simplecov but haven’t tried yet
-
simplecov isn’t quite working
-
simplecov mostly works when using rake test, but not autotest
-
only get 95% coverage with rake test and the lines that aren’t marked as tested really should have been
-
only get about 55% coverage with autotest, which I really don’t understand
-
-
-
‘pending’ doesn’t work the same
-
‘caller’ says that it is from a block now not method, so no name to use
-
using ‘skip’ instead
-
for some reason ‘skip’ doesn’t exist when running ‘rake test’, only ‘autotest’
-
-
still may have to deal with old mongrel version (there is a pre-release version)
-
or could just use webrick for the time being
-
No more ActiveRecord::Error and therefore no error :type which destroys my test method No more f.error_messages
Rewrite my custom validations
CSV Fixtures are deprecated and support will be removed in Rails 3.2.0. !!!!
json default output format no longer has the initial root key. config.active_record.include_root_in_json = false
-
resolve all commented out tests
-
change functional tests to only include fields on the forms
-
Is there a way to save without callbacks?
-
could be very useful in testing?
-
really trying to simplify all the testing as there are so many things that trigger other things. It is becoming difficult to test. Add one callback that creates something and all of the sudden 100 tests fail because some count is now off by 1.
-
-
fix user not logged in integration test. current_url doesn’t seem to match followed redirect. Also in CaseIntegrationTest
-
remove all method_missings from app and write explicit ones
-
may_*_required
-
-
reenable raising mass assignment error in environments/test.rb
-
controllers now raises MassAssignmentSecurity::Error
-
add a rescue to ignore it
-
only raises in dev and test, so no rescue
-
I could simply remove this … but seems scandalous
-
Raise exception on mass assignment protection for Active Record models
-
config.active_record.mass_assignment_sanitizer = :strict
-
I’d rather just silently ignore, but probably shouldn’t
-
-
datetime still accept partial? was some mention of error when not complete
-
think that there is/was some setting but can’t remember (seems to work)
-
-
remove soon to be deprecated code
-
remove any unused code
-
Keep or Remove HomexOutcome and custom HomeExposure callbacks????
-
begin taking advantage of the asset pipeline for merging js and css
-
baby steps using the manifest
-
-
make extract_fixtures_from rake task sort model attributes somehow
As this code is stored in publicly accessible repositories, DO NOT UNDER ANY CIRCUMSTANCES PUT ANY REAL SUBJECT DATA IN THE CODE!
Developers will certainly disagree with what to test and what not to test. I test A LOT. I test too much, perhaps. So much so that changing one thing can cause tests to fail way over there. I would rather that happen than having the app fail. I have tried to focus my tests so things are clearer, but there is certainly much more work to do in this arena. There are over 7000 tests and 27000 assertions as of right now. 23000 lines of code in the app and tests. It has become a daunting task to do much of anything without some repercussions.
Our Context [ … ] model method causes this …
ActionView::Template::Error: undefined method `[]' for ActionView::Context:Module
Added :: prefix where I use it to see if makes a difference
Due to our desire to have custom error messages, it was simplest to change the config/locales/en.yml file, which is used for the error messages to change the default format from “%{attribute} %{message}” to just “%{message}” and then add the “%{attribute}” for most and leave it out otherwise. A bit of a pain in the butt, but certainly effective. That being said, anytime a special error is explicitly added using something like “errors.add(:some_attribute,‘my custom message’)”, the custom message must include the attribute if desired as the system will not add it. This is actually working quite well.
This will create links like …
<a href=“/icf_master_tracker_updates/5806” class=“button” data-confirm=“Are you sure?” data-method=“delete” rel=“nofollow”>Destroy</a></td>
which contain a number of extra attributes. These extra attributes are not valid html. They are meant to be used by javascript confirmation, but they are still not valid html. So, since I’m not using them, I’ll replace these links with forms like the others.
railscasts.com/episodes/202-active-record-queries-in-rails-3
railscasts.com/episodes/215-advanced-queries-in-rails-3
railscasts.com/episodes/239-activerecord-relation-walkthrough
This is looking good.
def self.no_change_in_query(*args) scoped # just return scoped to do nothing apparently end
Repeated joins seem to be ignored. Yay!
> User.joins(:roles).joins(:roles).joins(:roles).joins(:roles).joins(:roles).joins(:roles).to_sql
=> "SELECT `users`.* FROM `users` INNER JOIN `roles_users` ON `roles_users`.`user_id` = `users`.`id` INNER JOIN `roles` ON `roles`.`id` = `roles_users`.`role_id`"
def self.with_role_name(role_name=nil)
( role_name.blank? ) ? scoped :
joins(:roles).where("roles.name".to_sym => role_name)
end
I do not like this as it takes precedence over your search unless you add ‘unscoped’ to your chain. It has caused me much grief. It caused grief with acts_as_list not properly scoping. It caused gried with my ZipCode limit not destroying all when I told it to destroy_all because it had “limit(10)” as part of its default_scope. Conclusion. DO NOT USE DEFAULT_SCOPE.
CANNOT have leading 0’s in a string that is being printed as a decimal as ruby thinks its octal and converts. Unless, of course, converting to octal was the intention.
>> sprintf("%06d","0001234")
=> "000668"
CANNOT have leading 0’s and include an 8 or 9 in a string that is being printed as a decimal as ruby thinks its octal and 8 and 9 are invalid octal characters so convert back to Integer first.
>> sprintf("%06d","0001280")
ArgumentError: invalid value for Integer: "0001280"
from (irb):24:in `sprintf'
from (irb):24
Don’t have validations that could raise errors that the user can’t do anything about.
When using validates_presence_of, use allow_blank => true on other validations to avoid multiple errors if blank.
When validating the presence of associations, validate on foreign key rather than association so that the error shows up correctly in view. Unfortunately, this will not guarantee that an association actually exists with that id as it would if the validation was on the association.
validates_presence_of :project ( will not raise error on project_id so in the view, the project_id selector won't have an error and so won't be highlighted like other errors will be.) validates_presence_of :project_id ( will not ensure that an actual project for the given id exists, but will show the error correctly in the view. )
I would like to have a validation that ensures that the _id is legitimate. I do not think that validates_associated is what I want. Having the both does what I want, but then the view will show “Project is blank” twice.
This seems to work …
validates_presence_of :project, :if => :project_id validates_presence_of :project_id
… but is a bit of a misnomer. The association validation will raise a ‘blank’ error, rather than a more appropriate ‘non-existant’ error, although the message could be altered.
Still some models and associations may need done due to priorities and the fact that some never end up in a view and the association may be problematic. (Aliquot,HomexOutcome,Patient)
Validating the content type of a paperclip attachment seems to be a bit tricky. A big problem seems to be the possibility of multiple content types. The paperclip validation seems to always fail.
validates_attachment_content_type :csv_file, :content_type => ["text/csv","text/plain","application/vnd.ms-excel"]
It seems like it almost expects the attachment to be ALL of the content types.
So, using the standard rails ‘validates inclusion of’ seems to work quite well.
validates_inclusion_of :csv_file_content_type, :in => ["text/csv","text/plain","application/vnd.ms-excel"], :allow_blank => true
For some reason, attempts to access any of the active scaffold controllers would end with …
Completed 500 Internal Server Error in 996ms ActionView::Template::Error (File to import not found or unreadable: active_scaffold_layout. Load path: / (in /my/ruby/gems/1.8/gems/active_scaffold-3.2.11/app/assets/stylesheets/active_scaffold.css.scss)):
while in production. This in-gem stylesheet contains @import which I gather do not work well with Rails 3, the asset pipeline and in-gem css. I have tried several different methods of pre-compiling these assets but it is all terribly inconvenient. Everything is fine on my development machine, even when I run it in production mode.
Pre-compiling seems to be the recommended way of doing things, but I don’t understand why. At this point, everything seems to be working.
So, I have copied the offending stylesheet from the active_scaffold gem to a local odms_active_scaffold.css.scss and changed the @imports to requires and everything seems to work a treat when this is used in the template.
Should this gem be updated and this core stylesheet change, our local copy will most likely need changed as well.
It appears that the “newer” classes know how to compare with the “older” ones, but not vice versa, which makes sense.
My comparisons in Sample should just have the “newer” class, the *_at field, first and compare it to the “older” *_on field. OR convert the DateTime to a date. Comparison of a Date to a DateTime will be confusing as it doesn’t just compare the date, it takes the time zone into account which isn’t 100% obvious. I think that it converts it to UTC before comparing. This could effectively make the comparison incorrect.
YES. Add a “to_date” to the _at fields.
>> Time.now => Tue Mar 06 20:00:06 -0800 2012 >> Time.now.class => Time >> Time.zone.now => Wed, 07 Mar 2012 03:54:31 UTC +00:00 >> Time.zone.now.class => ActiveSupport::TimeWithZone >> Time.zone.now > Date.today => true >> Date.today < Time.zone.now ArgumentError: comparison of Date with ActiveSupport::TimeWithZone failed from (irb):49:in `<' from (irb):49 >> Date.today < Time.now ArgumentError: comparison of Date with Time failed from (irb):50:in `<' from (irb):50
Also, even if the *_at field is given a Date value, it will be typecast to ActiveSupport::TimeWithZone so there is no terrible need to update all of the tests.
Unit tests are just simple tests, but are generally used to test the model. Functional tests are paired with a controller based on some simple conventions pulled from the test name. Integration tests are a little funky. They aren’t quite as popular and are very application specific. They aren’t tied to anything and and kinda meant to allow for testing a whole situation. I needed them for testing all of our javascript.
Initially, I tried using WebRat, but that does not test javascript. I then tried Capybara, with Selenium, which did test the javascript, but actually had to open a browser and only worked with Firefox, or Chrome with a special driver added. These were cool, but then I found Capybara-Webkit works as a simulated browser with Capybara. This is now the chosen setup for integration testing,
In these integration tests, the Capybara DSL, which has commands like ‘visit’, ‘find_field’, ‘fill_in’, etc., can be executed on the ‘page’ object, but is not necessary as these methods appear to be directly available to the test.
Be advised that some of the integration tests will intermittantly fail due to their threaded nature. I’ve added several ‘wait_until’ blocks which seems to work just fine most of the time. In addition, I upped the default time out. Now they rarely fail, but when they do, it is usually due to a timeout. This is mostly noticed when clicking a link or button and then expecting a change in the database. Sometimes there is none when there should be and then the assert_difference fails. As this behaviour is intermittant, it is a bit difficult to debug. On top of this, when these failures occur they seem to leave some stuff in the database, or put it in after the ROLLBACK, or something. Regardless, stuff gets left in the database and some completely unrelated tests may fail as this content is unexpected. Other than stopping the tests and running “rake db:test:prepare” to clear out the entire database, I have found no viable solution. I could re-write those tests that react poorly to this content, to be more relative than absolute, but this mayn’t always be possible. I could also possibly figure out how to stop the actual tainting of the database. I just upped the timeout to 6 seconds. In addition to the counts failing, existing data could include data which is required to be unique and may trigger a RecordInvalid as well. The key here is really to ensure that the database doesn’t get tainted.
Also note, that while has_css? can take an options hash with :visible, like … has_css?(‘div.phone_numbers div.historic’, :visible => false) … has_field? DOES NOT. It also does not correctly respond the the visible? method. If needing to test the presence and visibility of a form field, use find_field and then visible?.
ie. use this …
assert !page.find_field( "study_subject[subject_languages_attributes][2][other_language]").visible?
and not this …
assert page.has_field?( "study_subject[subject_languages_attributes][2][other_language]", :visible => false)
as the latter will be true if the field is there regardless of if it is visible
I noticed that occassionally something fails to rollback properly leaving some records in the database. This can and does causes some odd and confusing errors. Some of the difference tests are just wrong and it isn’t completely clear why. I believe that the integration tests are the initial cause as they are threaded. If something times out or errors in the integration test, things can get kinda funky. I have considered adding a couple destroy_alls, but that really slows things down, and I think would just undo on the rollback anyway. So this appears to be a framework issue. The only “fix” that I have found is to rerun “rake db:test:prepare” which wipes the test database clean. Just an FYI.
Also, recently rather than getting a clear link to a page showing html validation errors if and when they occur, an error similar to
Encoding::UndefinedConversionError: "\xE2" from ASCII-8BIT to UTF-8
is getting raised. This usually means that something on the resulting pages is invalid html. Good luck with that.
/config/api.yml ( probably removing ) /config/database.yml /config/initializers/mail.rb /config/initializers/secret_token.rb
-
port qt4-mac
-
gem webkit (not used anymore so should remove from Gemfile)
-
gem capybara
-
gem selenium-webdriver (not used anymore but seems required by others)
-
gem capybara-webkit
I have chosen to manage the input of other users by maintaining different repositories and merging as necessary. Using the same repo and creating branches was and is an option and I reserve the right to go that route if it looks better. initially ... git remote add magee [email protected]:magee/odms.git then when needed git fetch magee git merge --no-commit --no-ff magee/master
I installed the mysql2 gem and ran … to test the overall speed difference.
time rake odms_import:csv_data
Using just mysql
=> 4061.933u 70.565s 1:21:40.63 84.3% 0+0k 890+638io 0pf+0w
Using mysql2
=> 1878.205u 48.815s 44:48.35 71.6% 0+0k 851+497io 0pf+0w
Not surprisingly, mysql2 seems to be somewhere near twice as fast.
This seems to be working fine in development, testing and production. It is not necessary, but seems to be faster. If desired, all that would need to be done to downgrade to mysql is changing the database.yml adapter values.
Copyright © 2011 [Jake Wendt], released under the MIT license