Updating the Solr index with revised or new metadata

Periodically, the Solr index needs to be updated. Sometimes new records are created that need to be added. Sometimes existing records require updates because field content is revised.

Add or update records in the Solr index

1. Use the FTP program to upload your data document (we'll call it "data.json") to be indexed to the folder

/home/username/solr/solr-5.3.0/

*Note: Alternatively, if you have trouble uploading to this folder, you can upload to another folder and adapt the path in the next step.

2. Using Terminal, connect to the VPS as the root user:

$ ssh root@example.com

3. Change directories into the solr directory:

# cd /home/username/solr/solr-5.3.0
 
4. Index the data document into the Solr index by running the following Terminal command:

$ bin/post -c blacklight-core data.json

*Note: If you uploaded the data.json file to a folder other than the one given in step 1, you will need to adapt the path. For example, if you uploaded the file to the /home/username/ folder, the command will be:

$ bin/post -c blacklight-core ../../data.json

Deleting a single document from the Solr index

You may need to delete a single document from the Solr index at some time. This is a several step process if you use the Solr Admin UI.

1. Navigate to the Solar Admin UI in your web browser, select your core, and go to the Documents page.

2. Change the "Document Type" dropdown to "Solr Command" and enter the following text in the "Document(s)" field (change the "doc_1" text to the actual id of your document):

<delete><query>id:doc_1</query></delete>

3. Press the submit button to delete the document.

4. Enter the following text in the "Document(s)" field:

<commit/>

5. Press the submit button again to tell the Solr index to update the index (otherwise the changes will not show up in the Blacklight search interface)

6. Search for the document in the Blacklight search interface to see if the document has been successfully deleted.

Troubleshooting using the Rails production log

If you are still receiving the default Rails error screen, you can find the production log file to debug at

/home/username/public_html/test-app/log/production.log

Download this file and open it to see what errors are occurring

Restarting the Passenger app

To make sure that all configuration changes have taken effect, we need to restart the app through Passenger.

To restart the app through Passenger at any time, connect to the VPS and change directories to the app's home directory:

# cd /home/username/public_html/test-app

Next, run the following command:

# passenger-config restart-app

After you restart the app, go to your app's URL (example.com), and refresh the page to see if the changes have taken effect.

At this point, you hopefully see the Blacklight interface replace any errors that were on the screen in your web browser.


Previous post: Update Solr's solrconfig file

Next post: Troubleshooting using the Rails production log

Update Solr's solrconfig file

We also need to edit Solr's solrconfig.xml file to reflect the custom schema we are using. This also has to do with Solr communicating with Blacklight, which is not set up correctly because we used a separately installed Solr instance instead of installing it through Blacklight's installation.

1. Download and open up the solrconfig.xml file found at

/home/username/solr/solr-5.3.0/server/solr/blacklight-core/conf/solrconfig.xml

2. BL expects the <requestDispatcher> to have a handleSelect value of “true” because BL uses a “qt” equal to “search” or “document”. For Solr to recognize these as valid search operations, requestDispatcher needs to be changed. Find the <requestDispatcher> section in the solrconfig.xml file and replace it with:

<requestDispatcher handleSelect="true" >
  <requestParsers enableRemoteStreaming="false" multipartUploadLimitInKB="2048" />
</requestDispatcher>

3. Delete the requestHandler that has the name=“/select” attribute from the solrconfig.xml file. This will interfere with Blacklight looking for the “search” requestHandler.

***Make sure to DELETE it and not just comment it out. For some reason just commenting out doesn’t work ***

4. The requestHandlers Blacklight is expecting need to be created in the solrconfig.xml file. Using a solrconfig.xml file from a Blacklight installation that was installed with the --jettywrapper option, we can copy the “search” “document” and “advanced” request handlers into our solrconfig.xml file. These three handlers do not exist in solrconfig.xml by default.

****Because the catalog_controller.rb file can control the display of facet fields, we can go ahead and comment out any <str name="facet.field"> sections in the “search” requestHandler in solrconfig.xml****

****To control the fields that are searched when searching with a keyword, go to solrconfig.xml, go to the “search” requestHandler, and find the <str name="qf"> and <str name=“pf"> sections. These stand for “query fields” and “phrase fields”. Replace all of the fields currently listed in these two sections with the fields you want to be keyword-searched by BL. If you want to “boost” any of the fields you can add the “^” symbol followed by a number.****

Here are examples of the “search”, “document”, and “advanced” requestHandler sections, which will look something like this after editing to include our indexed field names, into the solrconfig.xml below the requestDispatcher section and just after the opening comments of the Request Handlers section:


<requestHandler name="search" class="solr.SearchHandler" default="true">
    <!-- default values for query parameters can be specified, these
         will be overridden by parameters in the request
      -->
     <lst name="defaults">
       <str name="defType">dismax</str>
       <str name="echoParams">explicit</str>
       <int name="rows">10</int>

       <str name="q.alt">*:*</str>
       <str name="mm">2&lt;-1 5&lt;-2 6&lt;90%</str>

       <!-- this qf and pf are used by default, if not otherwise specified by
            client. The default blacklight_config will use these for the
            "keywords" search. See the author_qf/author_pf, title_qf, etc
            below, which the default blacklight_config will specify for
            those searches. You may also be interested in:
            http://wiki.apache.org/solr/LocalParams
       -->

       <str name="qf">
        field1_txt
        field2_txt
        field3_txt
       </str>
       <str name="pf">
        field1_txt
        field2_txt
        field3_txt
       </str>
      
       <!--
       <str name="author_qf">
         author_unstem_search^200
         author_addl_unstem_search^50
         author_t^20
         author_addl_t
       </str>
       <str name="author_pf">
         author_unstem_search^2000
         author_addl_unstem_search^500
         author_t^200
         author_addl_t^10
       </str>
       <str name="title_qf">
         title_unstem_search^50000
         subtitle_unstem_search^25000
         title_addl_unstem_search^10000
         title_t^5000
         subtitle_t^2500
         title_addl_t^100
         title_added_entry_unstem_search^50
         title_added_entry_t^10
         title_series_unstem_search^5
         title_series_t
       </str>
       <str name="title_pf">
         title_unstem_search^500000
         subtitle_unstem_search^250000
         title_addl_unstem_search^100000
         title_t^50000
         subtitle_t^25000
         title_addl_t^1000
         title_added_entry_unstem_search^500
         title_added_entry_t^100
         title_series_t^50
         title_series_unstem_search^10
       </str>
       <str name="subject_qf">
         subject_topic_unstem_search^200
         subject_unstem_search^125
         subject_topic_facet^100
         subject_t^50
         subject_addl_unstem_search^10
         subject_addl_t
       </str>
       <str name="subject_pf">
         subject_topic_unstem_search^2000
         subject_unstem_search^1250
         subject_t^1000
         subject_topic_facet^500
         subject_addl_unstem_search^100
         subject_addl_t^10
       </str>
       -->
      
       <int name="ps">3</int>
       <float name="tie">0.01</float>

       <!-- NOT using marc_display because it is large and will slow things down for search results -->
       <str name="fl">
         id,
         score,
         field1_txt,
         field2_txt,
         field3_txt
       </str>

       <str name="facet">true</str>
       <str name="facet.mincount">1</str>
       <str name="facet.limit">40</str>
      
       <!--
       <str name="facet.field">subject_topic_facet</str>
       -->
      
       <str name="spellcheck">true</str>
       <str name="spellcheck.dictionary">default</str>
       <str name="spellcheck.onlyMorePopular">true</str>
       <str name="spellcheck.extendedResults">true</str>
       <str name="spellcheck.collate">false</str>
       <str name="spellcheck.count">5</str>

     </lst>
    <!-- In addition to defaults, "appends" params can be specified
         to identify values which should be appended to the list of
         multi-val params from the query (or the existing "defaults").
      -->
    <!-- In this example, the param "fq=instock:true" would be appended to
         any query time fq params the user may specify, as a mechanism for
         partitioning the index, independent of any user selected filtering
         that may also be desired (perhaps as a result of faceted searching).

         NOTE: there is *absolutely* nothing a client can do to prevent these
         "appends" values from being used, so don't use this mechanism
         unless you are sure you always want it.
      -->
    <!--
       <lst name="appends">
         <str name="fq">inStock:true</str>
       </lst>
      -->
    <!-- "invariants" are a way of letting the Solr maintainer lock down
         the options available to Solr clients.  Any params values
         specified here are used regardless of what values may be specified
         in either the query, the "defaults", or the "appends" params.

         In this example, the facet.field and facet.query params would
         be fixed, limiting the facets clients can use.  Faceting is
         not turned on by default - but if the client does specify
         facet=true in the request, these are the only facets they
         will be able to see counts for; regardless of what other
         facet.field or facet.query params they may specify.

         NOTE: there is *absolutely* nothing a client can do to prevent these
         "invariants" values from being used, so don't use this mechanism
         unless you are sure you always want it.
      -->
    <!--
       <lst name="invariants">
         <str name="facet.field">cat</str>
         <str name="facet.field">manu_exact</str>
         <str name="facet.query">price:[* TO 500]</str>
         <str name="facet.query">price:[500 TO *]</str>
       </lst>
      -->
    <!-- If the default list of SearchComponents is not desired, that
         list can either be overridden completely, or components can be
         prepended or appended to the default list.  (see below)
      -->
    <!--
       <arr name="components">
         <str>nameOfCustomComponent1</str>
         <str>nameOfCustomComponent2</str>
       </arr>
      -->
    <arr name="last-components">
      <str>spellcheck</str>
    </arr>
     
  </requestHandler>


<!-- for requests to get a single document; use id=666 instead of q=id:666 -->
  <requestHandler name="document" class="solr.SearchHandler" >
    <lst name="defaults">
      <str name="echoParams">all</str>
      <str name="fl">*</str>
      <str name="rows">1</str>
      <str name="q">{!raw f=id v=$id}</str> <!-- use id=666 instead of q=id:666 -->
    </lst>
  </requestHandler>
 
 
  <!--  For Advanced Search  -->
  <requestHandler name="advanced" class="solr.SearchHandler" >
    <lst name="defaults">
      <str name="defType">lucene</str>
      <str name="echoParams">explicit</str>
      <str name="sort">score desc, pub_date_sort desc, title_sort asc</str>  
      <str name="df">text</str>
      <str name="q.op">AND</str>
      <str name="qs">1</str>

      <!-- used for dismax query parser -->
      <str name="mm">1</str>
      <str name="ps">3</str>
      <float name="tie">0.01</float>
     
      <!-- for user query terms in author text box -->
      <str name="qf_author">
        author_unstem_search^200
        author_addl_unstem_search^50
        author_t^20
        author_addl_t
      </str>
      <str name="pf_author">
        author_unstem_search^2000
        author_addl_unstem_search^500
        author_t^200
        author_addl_t^10
      </str>
     
      <!-- for user query terms in title text box -->
      <str name="qf_title">
        title_unstem_search^50000
        subtitle_unstem_search^25000
        title_addl_unstem_search^10000
        title_t^5000
        subtitle_t^2500
        title_addl_t^100
        title_added_entry_unstem_search^50
        title_added_entry_t^10
        title_series_unstem_search^5
        title_series_t
      </str>
      <str name="pf_title">
        title_unstem_search^500000
        subtitle_unstem_search^250000
        title_addl_unstem_search^100000
        title_t^50000
        subtitle_t^25000
        title_addl_t^1000
        title_added_entry_unstem_search^500
        title_added_entry_t^100
        title_series_t^50
        title_series_unstem_search^10
      </str>
     
      <!-- for user query terms in subject text box -->
      <str name="qf_subject">
        subject_topic_unstem_search^200
        subject_unstem_search^125
        subject_topic_facet^100
        subject_t^50
        subject_addl_unstem_search^10
        subject_addl_t
      </str>
      <str name="pf_subject">
        subject_topic_unstem_search^2000
        subject_unstem_search^1250
        subject_t^1000
        subject_topic_facet^500
        subject_addl_unstem_search^100
        subject_addl_t^10
      </str>
     
      <!-- for user query terms in number text box -->
      <str name="qf_number">isbn_t</str>
     
      <!-- for user query terms in keyword text box -->
      <str name="qf_keyword">text</str>
      <str name="pf_keyword">text^10</str>
     
      <!-- NOT using marc_display because it is large and will slow things down for search results -->
      <str name="fl">
        id,
        score,
        author_display,
        author_vern_display,
        format,
        isbn_t,
        language_facet,
        lc_callnum_display,
        material_type_display,
        published_display,
        published_vern_display,
        pub_date,
        title_display,
        title_vern_display,
        subject_topic_facet,
        subject_geo_facet,
        subject_era_facet,
        subtitle_display,
        subtitle_vern_display,
        url_fulltext_display,
        url_suppl_display,
      </str>
     
      <str name="facet">true</str>
      <str name="facet.mincount">1</str>
      <str name="facet.limit">10</str>
      <str name="facet.field">format</str>
      <str name="facet.field">lc_1letter_facet</str>
      <str name="facet.field">lc_alpha_facet</str>
      <str name="facet.field">lc_b4cutter_facet</str>
      <str name="facet.field">language_facet</str>
      <str name="facet.field">pub_date</str>
      <str name="facet.field">subject_era_facet</str>
      <str name="facet.field">subject_geo_facet</str>
      <str name="facet.field">subject_topic_facet</str>
     
      <str name="spellcheck">true</str>
      <str name="spellcheck.dictionary">subject</str>
      <str name="spellcheck.onlyMorePopular">true</str>
      <str name="spellcheck.extendedResults">true</str>
      <str name="spellcheck.collate">false</str>
      <str name="spellcheck.count">5</str>
    </lst>
    <arr name="last-components">
      <str>spellcheck</str>
    </arr>
  </requestHandler>

5. In addition to the “search” “document” and “advanced” requestHandlers, the Blacklight solrconfig.xml file has additional requestHandlers that are not in the default solrconfig.xml file. Copy the following requestHandler sections from the Blacklight solrconfig.xml to the custom solrconfig.xml file and place them above the “search” requestHandler:

<requestHandler name="/replication" class="solr.ReplicationHandler" startup="lazy" />
<requestHandler name="standard" class="solr.StandardRequestHandler" />
<requestHandler name="/update" class="solr.UpdateRequestHandler"  />
<requestHandler name="/admin/" class="org.apache.solr.handler.admin.AdminHandlers" />
    
<requestHandler name="/admin/ping" class="solr.PingRequestHandler">
   <lst name="invariants">
     <str name="q">solrpingquery</str>
   </lst>
   <lst name="defaults">
     <str name="echoParams">all</str>
   </lst>
</requestHandler>

6. We also need to copy the <searchComponent name="spellcheck" class="solr.SpellCheckComponent"> section from an existing Blacklight solrconfig.xml into our solrconfig.xml file and place it after the last existing searchComponent section:

<!-- Spell Check

        The spell check component can return a list of alternative spelling
        suggestions. 

        http://wiki.apache.org/solr/SpellCheckComponent
     -->
  <searchComponent name="spellcheck" class="solr.SpellCheckComponent">

    <str name="queryAnalyzerFieldType">textSpell</str>

    <!-- Multiple "Spell Checkers" can be declared and used by this
         component
      -->

    <!-- a spellchecker built from a field of the main index, and
         written to disk
      -->
    <lst name="spellchecker">
      <str name="name">default</str>
      <str name="field">spell</str>
      <str name="spellcheckIndexDir">./spell</str>
      <str name="buildOnOptimize">true</str>
    </lst>
    <lst name="spellchecker">
      <str name="name">author</str>
      <str name="field">author_spell</str>
      <str name="spellcheckIndexDir">./spell_author</str>
      <str name="accuracy">0.7</str>
      <str name="buildOnOptimize">true</str>
    </lst>
    <lst name="spellchecker">
      <str name="name">subject</str>
      <str name="field">subject_spell</str>
      <str name="spellcheckIndexDir">./spell_subject</str>
      <str name="accuracy">0.7</str>
      <str name="buildOnOptimize">true</str>
    </lst>
    <lst name="spellchecker">
      <str name="name">title</str>
      <str name="field">title_spell</str>
      <str name="spellcheckIndexDir">./spell_title</str>
      <str name="accuracy">0.7</str>
      <str name="buildOnOptimize">true</str>
    </lst>

    <!-- a spellchecker that uses a different distance measure -->
    <!--
       <lst name="spellchecker">
         <str name="name">jarowinkler</str>
         <str name="field">spell</str>
         <str name="distanceMeasure">
           org.apache.lucene.search.spell.JaroWinklerDistance
         </str>
         <str name="spellcheckIndexDir">spellcheckerJaro</str>
       </lst>
     -->

    <!-- a spellchecker that use an alternate comparator

         comparatorClass be one of:
          1. score (default)
          2. freq (Frequency first, then score)
          3. A fully qualified class name
      -->
    <!--
       <lst name="spellchecker">
         <str name="name">freq</str>
         <str name="field">lowerfilt</str>
         <str name="spellcheckIndexDir">spellcheckerFreq</str>
         <str name="comparatorClass">freq</str>
         <str name="buildOnCommit">true</str>
      -->

    <!-- A spellchecker that reads the list of words from a file -->
    <!--
       <lst name="spellchecker">
         <str name="classname">solr.FileBasedSpellChecker</str>
         <str name="name">file</str>
         <str name="sourceLocation">spellings.txt</str>
         <str name="characterEncoding">UTF-8</str>
         <str name="spellcheckIndexDir">spellcheckerFile</str>
       </lst>
      -->
  </searchComponent>

7. Use the FTP program to replace the solrconfig.xml file on the server with the one you just edited.

8. Go to the Terminal window in which you are logged in as root and navigate to

/home/username/solr/solr-5.3.0

9. Stop and restart the solar service again so the new solrconfig.xml edits will take effect:

# bin/solr stop -all
# bin/solr restart


Previous post: Update Blacklight's catalog_controller file

Next post:

Update Blacklight's catalog_controller file

We need to update the Blacklight app's catalog_controller file to match the schema fields we added to the Solr schema configuration file.

1. With a FTP program, download Blacklight’s CatalogController file found at

/home/username/public_html/test-app/app/controllers/catalog_controller.rb

2. Open it in a text editor and begin editing by making the config.default_solr_params array look like:

config.default_solr_params = {
  qt: 'search',
  rows: 10
}

3. Uncomment the following line (by deleting the hash symbol) so it looks like:

config.per_page = [10,20,50,100]

You can change the numbers inside the array if you want to change the options users see for the number of results shown on the screen.

4. Also uncomment the following array so it looks like:

config.default_document_solr_params = {
      qt: 'document',
      ## These are hard-coded in the blacklight 'document' requestHandler
       fl: '*',
       rows: 1,
       q: '{!term f=id v=$id}'
    }

5. Change the following lines:

# solr field configuration for search results/index views
config.index.title_field = 'title_display'
config.index.display_type_field = 'format'

# solr field configuration for document/show views
#config.show.title_field = 'title_display'
#config.show.display_type_field = 'format'

to:

# solr field configuration for search results/index views
config.index.title_field = 'field1_txt'
#config.index.display_type_field = 'format'

# solr field configuration for document/show views
config.show.title_field = 'field1_txt'
#config.show.display_type_field = 'format'

where "field1_txt" is the name of whichever field you want to show as the title of the individual record.

6. Edit the existing config.add_facet_field lines in the config.add_facet_field section to use your schema's field names: 

config.add_facet_field 'field1Facet', label: 'Field 1', limit: 10
config.add_facet_field 'field2Facet', label: 'Field 2', limit: 15


You can also add label names for the facets and define limits on teh number of facet options displayed. For further customization of the facet fields, see the Blacklight site.

7. Also, to ensure facet fields are shown, make sure that the following line is not commented out:

config.add_facet_fields_to_solr_request!

8. Edit the existing config.add_index_field lines and add fields that you want to display in the results list to the config.add_index_field section:

config.add_index_field 'field2_txt', label: 'Field 2'
config.add_index_field 'field3_txt', label: 'Field 3', link_to_search: 'field3Facet'


You can use the link_to_search helper to create a search shortcut. For further customization of this section, see the Blacklight online guide.

9. Edit the existing config.add_show_field lines and add fields to the config.add_show_field section that you want to see displayed in the individual result page:

config.add_show_field 'field2_txt', label: 'Field 2'
config.add_show_field 'field3_txt', label: 'Field 3'


10. Comment out all lines of the following sections by placing hashtag symbols at the beginning of each line:

config.add_search_field('title') do |field|
  # solr_parameters hash are sent to Solr as ordinary url query params.
  field.solr_parameters = { :'spellcheck.dictionary' => 'title' }

  # :solr_local_parameters will be sent using Solr LocalParams
  # syntax, as eg {! qf=$title_qf }. This is neccesary to use
  # Solr parameter de-referencing like $title_qf.
  # See: http://wiki.apache.org/solr/LocalParams
  field.solr_local_parameters = {
    qf: '$title_qf',
    pf: '$title_pf'
  }
end

config.add_search_field('author') do |field|
  field.solr_parameters = { :'spellcheck.dictionary' => 'author' }
  field.solr_local_parameters = {
    qf: '$author_qf',
    pf: '$author_pf'
  }
end

# Specifying a :qt only to show it's possible, and so our internal automated
# tests can test it. In this case it's the same as
# config[:default_solr_parameters][:qt], so isn't actually neccesary.
config.add_search_field('subject') do |field|
  field.solr_parameters = { :'spellcheck.dictionary' => 'subject' }
  field.qt = 'search'
  field.solr_local_parameters = {
    qf: '$subject_qf',
    pf: '$subject_pf'
  }
end

11. Edit the existing config.add_sort_field lines to use fieldnames that you would like the user to be able to sort the results:

config.add_sort_field 'score desc, field1_txt asc', label: 'Relevance'
config.add_sort_field 'field1_txt asc', label: 'Field 1'

12. Save the catalog_controller.rb file and use the FTP program to replace the server copy with the copy you just edited.


Previous post: Update Solr schema configuration

Next post: Update Solr's solrconfig file

Update Solr schema configuration

We need to update the default Solr configuration to work with our data. We also need to upload some initial data into the Solr index before the Rails app and Passenger will display the Blacklight app interface on the screen.

Creating the new Solr core

1. Using Terminal, connect to the VPS as the root user:

$ ssh root@example.com

2. Change directories into the solr directory:

# cd /home/username/solr/solr-5.3.0

3. In a web browser, check to see if Solr is already running by going to

example.com:8983

If it is, skip the following start up step. If it is not running, start the Solr service with the following command in Terminal:

# bin/solr start -noprompt

4. Create a Solr core for the Blacklight data using the basic_configs instead of the default data_driven_schema_configs (these can be found at solr-5.3.0/server/solr/configsets and are copied in during the core creation):

# bin/solr create -c blacklight-core -d basic_configs

5. The new core should now have a directory at

/home/username/solr/solr-5.3.0/server/solr/blacklight-core

6. Refresh the Solr admin UI in the web browser to confirm the new core is created and accessible.

Updating the default Solr schema

7. The blacklight-core collection in Solr is currently using the default scheme found at

/home/username/solr/solr-5.3.0/server/solr/blacklight-core/conf/schema.xml

The Blacklight interface is not set up to use this schema. Edits to both the Blacklight settings and the Solr schema will be necessary.

8. Download the schema.xml file using a FTP program and open it in a text editor.

9. I explicitly defined a number of fields by adding the lines such as the following below the <field name=“id” … /> line:

<field name="field1_txt" type="text_general" indexed="true" stored="true" multiValued="true"/>
<field name="field1Facet" type="string" indexed="true" stored="false" multiValued="true"/>
  
<field name="field2_txt" type="text_general" indexed="true" stored="true" multiValued="false"/>

As the fields above indicate, some of the fields I defined were multivalued="true" and some were multivalued="false". This depended on whether the field would be used as a sort field because it appears sort fields can only be multivalued="false".

Create as many field names as you need to fill out your schema.

10. Go down in the schema.xml file below the <uniqueKey> line and add the following copyFields:

<copyField source="field1_txt" dest="field1Facet"/>

This copies the value of the source field into an additional field "as is" instead of being reformatted as part of the indexing process. This is necessary for the Facet options to show up correctly, so do this for any fields you plan to use as facets.

11. Save the schema.xml file and use the FTP program to replace the copy on the server with the copy you just edited.

12. In terminal, restart the solar service with the following two commands:

$ bin/solr stop -all
$ bin/solr restart

13. Use the FTP program to upload your data document (we'll call it "data.json") to be indexed to the folder

/home/username/solr/solr-5.3.0/

14. Index the data document into the Solr index by running the following Terminal command:

$ bin/post -c blacklight-core data.json

15. Refresh the Solr Admin UI in the web browser to confirm the document was indexed (there should be many more docs in the index now)

Errors will still appear on the Blacklight interface because Blacklight’s CatalogController is looking for certain fields that are not in the schema.


Previous post: Determine Ruby command for Passenger and update configuration files

Next post: Update Blacklight's catalog_controller file