<?xml version="1.0" encoding="utf-8"?>
<html>
An experiment by: Richard Jones.
...
The current browse configuration in the dynamic browse looks like this:
| Panelcode |
|---|
webui.browse.index.1 = dateissued:dc.date.issued:date:full webui.browse.index.2 = author:dc.contributor.*:text:single webui.browse.index.3 = title:dc.title:title:full webui.browse.index.4 = subject:dc.subject.*:text:single webui.browse.index.5 = dateaccessioned:dc.date.accessioned:date:full # and some extra examples ones # webui.browse.index.6 = type:dc.type:text:single # webui.browse.index.7 = itemstatus:icadmin.status:text:single |
it is therefore of the form:
| Panelcode |
|---|
webui.browse.index.<n> = <index name>:<metadata field>:<data type>: |
The proposed addition is the following:
| Panelcode |
|---|
webui.browse.sort-option.<x> = <sort name>:<metadata field>:<data type> |
This will allow the browse index mechanism to generate tables which contain enough information to sort results without having to instantiate any Item objects at all. This will make browsing faster and enable browses such as "all items by the author XXX" to be paginated successfully.
Therefore, new configuration will look like this (which will have the same effect as the current default):
| Code Block |
|---|
# Set the options for what can be sorted by
|
| Panel |
Set the options for what can be sorted bywebui.browse.sort-option.1 = title:dc.title:text webui.browse.sort-option.2 = date:dc.date.issued:date |
Data Model
The next stage is to rewrite the BrowseIndex class to be able to pick up this configuration for each of the indexes mentioned above. It will then create a number of Browse tables (when necessary):
...
This is similar to the existing browse and the dynamic browse patch, but the critical difference is in the table structure. index_<n> will be structured thus:
| Panelcode |
|---|
item_id: int not null FK // the item id value:text // the text value of the core browse value text // the text value of the core browse value sort_value: text // the normalised text value of the core browse value sort_<x> // x number of columns corresponding to the normalised value of the sort-options defined above |
Progress Update: 21-11-2006, 13:40 GMT
...
- obtain all the BrowseItem objects
- obtain all the BrowseIndex objects (which will, in turn, contain the SortOption objects)
- for each BrowseItem object
- for each BrowseIndex object
- delete the existing index data for the BrowseItem
- get the "value" metadata from the BrowseItem
- for each SortOption object
- get the "sort" metadata from the BrowseItem
- for each metadata value (primary index)
- write a line into the database with the index value and sort option values (normalised)
- commit the transaction
- for each BrowseIndex object
- for each BrowseIndex object
- delete all item ids that are in the index table but not the item table
| Panelcode |
|---|
NOTE: a potential problem arises. The "sort" fields need to be singular, while the "value" field can be multiple. That is, an item may have more than one author as the value to browse on, but may not have more than one title as the value to sort on. In the cases where an item has more than one value in the sort metadata, the code will select only the first value that is returned. This is a caveat that people configuring their system will need to be aware of |
Progress Update: 21-11-2006, 14:50 GMT
Against all belief, the indexing code appears to already be working. It as much as 0.1 seconds slower per item in a small database, which may need to be addressed (see TODO below). This code touches the following files: IndexBrowse.java, SortOption.java, BrowseException.java. No new files were necessary. I have included below two screen dumps of my test database, as an example as to what I am currently seeing in the indices:
| Code Block |
|---|
# select * from |
| Panel |
select * from index_2;id | id | item_id | value | sort_value | sort_2 | sort_1 ----+---------+----------------+----------------+----------------------+---------- 1 | 1 | Jones, Richard | jones, richard | - 1 | 1 | Jones, Richard | jones, richard | 2006-11-16t17:08:11z | submit1 2 | 2 | Jones, Richard | jones, richard | 1 2 | 2 | Jones, Richard | jones, richard | 2006-11-16t17:08:42z | submit2 3 | 3 | Jones, Richard | jones, richard | 2 3 | 3 | Jones, Richard | jones, richard | 2006-11-16t17:09:05z | submit3 4 | 4 | Jones, Richard | jones, richard | 3 4 | 4 | Jones, Richard | jones, richard | 2006-11-16t17:09:26z | submit4 5 | 5 | Jones, Richard | jones, richard | 4 5 | 5 | Jones, Richard | jones, richard | 2006-11-16t17:09:52z | submit5 6 | 6 | Jones, Richard | jones, richard | 5 6 | 6 | Jones, Richard | jones, richard | 2006-11-16t17:10:18z | submit6 7 | 7 | Jones, Richard | jones, richard | 6 7 | 7 | Jones, Richard | jones, richard | 2006-11-16t17:10:43z | submit7 8 | 8 | Jones, Richard | jones, richard | 7 8 | 8 | Jones, Richard | jones, richard | 2006-11-16t17:11:08z | submit8 9 | 9 | Jones, Richard | jones, richard | 8 9 | 9 | Jones, Richard | jones, richard | 2006-11-16t17:11:30z | submit9 10 | 10 | Jones, Richard | jones, richard | 9 10 | 10 | Jones, Richard | jones, richard | 2006-11-16t17:11:56z | submit10 11 | 11 | Jones, Richard | jones, richard | 10 11 | 11 | Jones, Richard | jones, richard | 2006-11-16t17:12:19z | submit11 12 | 12 | Jones, Richard | jones, richard | 11 12 | 12 | Jones, Richard | jones, richard | 2006-11-16t17:12:45z | submit12 13 | 13 | Jones, Richard | jones, richard | 12 13 | 13 | Jones, Richard | jones, richard | 2006-11-16t17:13:09z | submit13 14 | 14 | Jones, Richard | jones, richard | 13 14 | 14 | Jones, Richard | jones, richard | 2006-11-16t17:13:31z | submit14 15 | 15 | Jones, Richard | jones, richard | 14 15 | 15 | Jones, Richard | jones, richard | 2006-11-16t17:13:52z | submit15 16 | 16 | Jones, Richard | jones, richard | 15 16 | 16 | Jones, Richard | jones, richard | 2006-11-16t17:14:16z | submit16 17 | 17 | Jones, Richard | jones, richard | 16 17 | 17 | Jones, Richard | jones, richard | 2006-11-16t17:14:37z | submit17 18 | 18 | Jones, Richard | jones, richard | 17 18 | 18 | Jones, Richard | jones, richard | 2006-11-16t17:14:58z | submit18 19 | 19 | Jones, Richard | jones, richard | 18 19 | 19 | Jones, Richard | jones, richard | 2006-11-16t17:15:19z | submit19 20 | 20 | Jones, Richard | jones, richard | 19 20 | 20 | Jones, Richard | jones, richard | 2006-11-16t17:15:42z | submit20 21 | 21 | Jones, Richard | jones, richard | 20 21 | 21 | Jones, Richard | jones, richard | 2006-11-16t17:16:02z | submit21 22 | 22 | Jones, Richard | jones, richard | 21 22 | 22 | Jones, Richard | jones, richard | 2006-11-16t17:17:24z | submit 22 (22 rows) |
- select * from index_1;
Panel Code Block id |id |item_id | value | sort_value | sort_2 | sort_1
-----+---------+----------------------+----------------------+----------------------+-----------
1 | 1 |1 | 1 | 2006-11-16T17:08:11Z | 2006-11-16t17:08:11z | 2006-11-16t17:08:11z | submit
1
2 | 2 |1 2 | 2 | 2006-11-16T17:08:42Z | 2006-11-16t17:08:42z | 2006-11-16t17:08:42z | submit
2
3 | 3 |2 3 | 3 | 2006-11-16T17:09:05Z | 2006-11-16t17:09:05z | 2006-11-16t17:09:05z | submit
3
4 | 4 |3 4 | 4 | 2006-11-16T17:09:26Z | 2006-11-16t17:09:26z | 2006-11-16t17:09:26z | submit
4
5 | 5 |4 5 | 5 | 2006-11-16T17:09:52Z | 2006-11-16t17:09:52z | 2006-11-16t17:09:52z | submit
5
6 | 6 |5 6 | 6 | 2006-11-16T17:10:18Z | 2006-11-16t17:10:18z | 2006-11-16t17:10:18z | submit
6
7 | 7 |6 7 | 7 | 2006-11-16T17:10:43Z | 2006-11-16t17:10:43z | 2006-11-16t17:10:43z | submit
7
8 | 8 |7 8 | 8 | 2006-11-16T17:11:08Z | 2006-11-16t17:11:08z | 2006-11-16t17:11:08z | submit
8
9 | 9 |8 9 | 9 | 2006-11-16T17:11:30Z | 2006-11-16t17:11:30z | 2006-11-16t17:11:30z | submit
9
10 | 10 |9 10 | 10 | 2006-11-16T17:11:56Z | 2006-11-16t17:11:56z | 2006-11-16t17:11:56z | submit
10
11 | 11 |10 11 | 11 | 2006-11-16T17:12:19Z | 2006-11-16t17:12:19z | 2006-11-16t17:12:19z | submit
11
12 | 12 |11 12 | 12 | 2006-11-16T17:12:45Z | 2006-11-16t17:12:45z | 2006-11-16t17:12:45z | submit
12
13 | 13 |12 13 | 13 | 2006-11-16T17:13:09Z | 2006-11-16t17:13:09z | 2006-11-16t17:13:09z | submit
13
14 | 14 |13 14 | 14 | 2006-11-16T17:13:31Z | 2006-11-16t17:13:31z | 2006-11-16t17:13:31z | submit
14
15 | 15 |14 15 | 15 | 2006-11-16T17:13:52Z | 2006-11-16t17:13:52z | 2006-11-16t17:13:52z | submit
15
16 | 16 |15 16 | 16 | 2006-11-16T17:14:16Z | 2006-11-16t17:14:16z | 2006-11-16t17:14:16z | submit
16
17 | 17 |16 17 | 17 | 2006-11-16T17:14:37Z | 2006-11-16t17:14:37z | 2006-11-16t17:14:37z | submit
17
18 | 18 |17 18 | 18 | 2006-11-16T17:14:58Z | 2006-11-16t17:14:58z | 2006-11-16t17:14:58z | submit
18
19 | 19 |18 19 | 19 | 2006-11-16T17:15:19Z | 2006-11-16t17:15:19z | 2006-11-16t17:15:19z | submit
19
20 | 20 |19 20 | 20 | 2006-11-16T17:15:42Z | 2006-11-16t17:15:42z | 2006-11-16t17:15:42z | submit
20
21 | 21 |20 21 | 21 | 2006-11-16T17:16:02Z | 2006-11-16t17:16:02z | 2006-11-16t17:16:02z | submit
21
22 | 22 |21 22 | 22 | 2006-11-16T17:17:24Z | 2006-11-16t17:17:24z | 2006-11-16t17:17:24z | submit 22
(22 rows)
TODO: the scale problem is at least in part because the sort values are obtained for each item for each browse index, which is an
unnecessary amount of work. Refactoring should sort this out, but it remains as-is for the moment because it slipped easily into
existing code
...
The following are variables that will need to be passed into the Browse engine in order for appropriate results to be returned: *
- type: the type of browse being undertaken. This will be used to identify the Browse Index from the config
...
- sortBy: which of the available sort options in config is to be sorted by
...
- order: which way to interpret the sortBy. ASC or DESC
...
- value: a specific value to browse upon. For example "Jones, Richard" to view all items where I am the author (in conjunction with type=author, of course)
...
- resultsperpage: number of results to display on the page at any one time
...
- community: the community we are browsing in
...
- collection: the collection we are browsing in
...
- next: the id of the item to be at the top of the "next" page
...
- prev: the id of the item to be at the top of the "previous" page
...
- focus: the target point in the listing to point the browse. This will be utilised by the paging system
...
- year: the year to use as a focus in date browse
...
- month: the month to use as a focus in date browse
...
- startsWith: the characters to use for a stem search. Will be used with the focus
...
- vfocus: the string to form the focus for single browse contexts added 29-11-2006
NOTE: "next" and "prev" are not clearly defined as to what the best way to obtain them is, and exactly what their relationship to "focus" is. It may be that "next" and "prev" are only used in the Servlet/UI layer to represent the "focus" for the next and previous functionality.
...
Obtain the results for a given value or focus:
| Panelcode |
|---|
SELECT * FROM <index> WHERE sort_value [<<value> AND |=] [<value> | _focus_] [AND collection_id = _collection_ AND ] [AND community_id = _community_ ORDER BY <sortBy> <order> LIMIT <resultsperpage + 1> ]
ORDER BY <sortBy> <order>
LIMIT <resultsperpage + 1>
|
So if "focus" is used to tell us which "next" or "prev" we should be looking at, then we may need to be able to dispense with them all together. <index> is updated to refer to the relevant table name (whether it is index_<n>, collection_index_<n> or community_index_<n>), and if the sort terms are correctly prepared then the simple comparators should be enough to ensure that we get everything in the desired order.
In order to output the string "Results A - B of C" the following are requried (B = A + <resultsperpage>):
A:
| Panelcode |
|---|
SELECT COUNT  (*) FROM <index> WHERE value [<|_] <focus> |
Here the "focus" must be implicitly defined for every request so that this query always returns, although sometimes will return 0. That means that after the SELECT above we must at least always assign the first result to be the focus if it has not already been defined).
C:
| Panelcode |
|---|
SELECT COUNT (*) FROM<index> WHERE value = <index> [WHERE value = _value_] |
Ongoing programmer notes
- sortBy is an int parameter, indicating which sort field to use. sortBy = 0 will therefore be sort by the index value
- focus is either: a value pulled from the UI top navigation; a specific item id to browse to. This strikes me that it might need to be divided into two parts
- I have made an executive decision that focus will refer only to item ids. Everything else must go through "value" or "starts_with"
- if order = ASC value comparator = >, if order = DESC value comparator = <. How about =? This is when a value is supplied, in which case the comparator is applicable to the sortBy field instead
- QUESTION: do we do our comparisons for browse on the "value" or the "sort_value" fields. Since "sort_value" has some sort of normalisation applied to it we must either normalise the request and compare it to that or not normalise the request and compare it to the "value" column
...
It has become necessary, on implementation, to modify the first SQL query given above, and add an additional query to satisfy paging with item focusses:
The first query becomes:
| Panelcode |
|---|
SELECT * FROM <index> WHERE sort_value [<<value> AND |=] [<value> | _focus-value_] [AND collection_id = _collection_ AND ] [AND community_id = _community_ ORDER BY <sortBy> <order> LIMIT <resultsperpage + 1> ]
ORDER BY <sortBy> <order>
LIMIT <resultsperpage + 1>
|
Just a minor change to indicate that the sort_value is not the <focus> as previously indicated, but the value of the <focus> id in the relevant context. This means we must add the following SQL:
| Panelcode |
|---|
SELECT sort_value FROM <index> WHERE item_id = <focus> |
This seems like a slightly dodgy solution, but this is almost exactly how the current browse mechanism does it. I will implement it this way, and revisit later if there are problems.
...
type = title
order = ASC
focus = -1
value = null
rrp = 21
startsWith = null
sortBy = 0
| Panelcode |
|---|
SELECT * FROM index_1 ORDER BY sort_value ASC LIMIT 21 |
2) Browse page 3 of the author list
type = author
order = ASC
focus = 32 (random item id)
value = null
rrp = 21
startsWith = null
sortBy = 0
| Panelcode |
|---|
SELECT * FROM index_2 WHERE sort_value > [focusvalue ORDER BY value] ORDER BY sort_value ASC LIMIT 22 |
3) Browse page 2 of author Jones, Richard, ordering by title
type = author
order = ASC
focus = 11 (random item id)
value = Jones, Richard
rrp = 21
startsWith = null
sortBy = 1 (title field)
| Panelcode |
|---|
SELECT * FROM index_2 WHERE sort_1 > [focusvalue AND value] AND sort_value = 'jones, richard' ORDER BY sort_1 ASC LIMIT 22 |
Ongoing programmers notes
...
Ongoing programmers notes
...
- Here's an oddity which rings a bell from the last time I looked at the Browse code. It seems that the result of a SELECT COUNT query, despite being a number can't be retrieved using the TableRow.getIntColumn() method. I'm trying TableRow.getStringColumn and Integer.parseInt, but I have a horrible feeling that won't work either.
- Nope, as anticipated:
| Panelcode |
|---|
Exception: java.lang.IllegalArgumentException: Value for number is not aninteger at integer at org.dspace.storage.rdbms.TableRow.getIntColumn(TableRow.java:162) |
Exception:
| Panelcode |
|---|
java.lang.IllegalArgumentException: Value is not anstring at string at org.dspace.storage.rdbms.TableRow.getStringColumn(TableRow.java:244) |
- a spot of experimentation suggests that it can be got hold of as a "long"
- Initial experiments with the browse code are looking positive. At least there are no errors coming out of the SQL and result sets containing actual data are being returned. Of course, we don't know for certain that it is the correct data yet. One problem that we need to address is some sort of entity object to cover the display end of things. As you may know, the current configuration for browse listings looks like this:
| Panelcode |
|---|
webui.itemlist.columns = dc.date.issued(date), dc.title, dc.contributor.* |
In order to make our debug better, it would be handy to have this wrapped up in an object that could do the configuration, rather than buried in the ItemListTag.java which is where it is just now
...
I have got code that "works" in the minimal sense of the word, which appears to give us useful results (still to confirm that they are correct in each context, see below). Adding a toString() method to the BrowseInfo, I have dumped browse results to the screen for debugging. Below I have pasted the browse results corresponding to index_2 and index_1 in my dev box, which represent the tables shown further up this page. The work has touched the following files: BrowseInfo.java, DatabaseManager.java (for extra debug), dspace-web.xml, and created the following new files: BrowserServlet.java, BrowserScope.java, ItemListConfig.java.
| Panelcode |
|---|
BrowseInfo String Representation: Browsing 0 to 22 of 22|| Listing over 3 columns: dc.date.issued,dc.title.null,dc.contributor.*|| Item {{Item ID: 22 :: [dc.date.issued:2006-11-16T17.17.24Z][dc.title.null:submit 22][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 21 :: [dc.date.issued:2006-11-16T17.16.02Z][dc.title.null:submit 21][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 20 :: [dc.date.issued:2006-11-16T17.15.42Zsubmit 20 ][dc.title.null:submit 20][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 19 :: [dc.date.issued:2006-11-16T17.15.19Z][dc.title.null:submit 19][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 18 :: [dc.date.issued:2006-11-16T17.14.58Z][dc.title.null:submit 18][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 17 :: [dc.date.issued:2006-11-16T17.14.37Z][dc.title.null:submit 17][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 16 :: [dc.date.issued:2006-11-16T17.14.16Z][dc.title.null:submit 16][dc.contributor.*:Jones,Richard Item ID: 15 : Richard]}} {{Item ID: 15 :: [dc.date.issued:2006-11-16T17.13.52Z][dc.title.null:submit 15][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 14 :: [dc.date.issued:2006-11-16T17.13.31Z][dc.title.null:submit 14][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 13 :: [dc.date.issued:2006-11-16T17.13.09Z][dc.title.null:submit 13][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 12 :: [dc.date.issued:2006-11-16T17.12.45Z][dc.title.null:submit 12][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 11 :: [dc.date.issued:2006-11-16T17.12.19Zsubmit 11Jones, Richard Item ID: 10 :: ][dc.title.null:submit 11][dc.contributor.*:Jones, Richard]}} {{Item ID: 10 :: [dc.date.issued:2006-11-16T17.11.56Z][dc.title.null:submit 10][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 9 :: [dc.date.issued:2006-11-16T17.11.30Z][dc.title.null:submit 9][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 8 :: [dc.date.issued:2006-11-16T17.11.08Z][dc.title.null:submit 8][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 7 :: [dc.date.issued:2006-11-16T17.10.43Z][dc.title.null:submit 7][dc.contributor.*:Jones,Richard Item ID: 6 : Richard]}} {{Item ID: 6 :: [dc.date.issued:2006-11-16T17.10.18Z][dc.title.null:submit 6][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 5 :: [dc.date.issued:2006-11-16T17.09.52Z][dc.title.null:submit 5][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 4 :: [dc.date.issued:2006-11-16T17.09.26Z][dc.title.null:submit 4][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 3 :: [dc.date.issued:2006-11-16T17.09.05Z][dc.title.null:submit 3][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 2 :: [dc.date.issued:2006-11-16T17.08.42Zsubmit 2Jones, Richard Item ID: 1 :: ][dc.title.null:submit 2][dc.contributor.*:Jones, Richard]}} {{Item ID: 1 :: [dc.date.issued:2006-11-16T17.08.11Z][dc.title.null:Submit 1][dc.contributor.*:Jones, Richard]}}|| |
BrowseInfo String Representation: Browsing 0 to 22 of 22||
| Panelcode |
|---|
Listing over 3 columns: dc.date.issued,dc.title.null,dc.contributor.*|| Item {{Item ID: 1 :: [dc.date.issued:2006-11-16T17.08.11Z][dc.title.null:Submit 1][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 2 :: [dc.date.issued:2006-11-16T17.08.42Z][dc.title.null:submit 2][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 3 :: [dc.date.issued:2006-11-16T17.09.05Z][dc.title.null:submit3Jones, Richard Item ID: 4 :: 3][dc.contributor.*:Jones, Richard]}} {{Item ID: 4 :: [dc.date.issued:2006-11-16T17.09.26Z][dc.title.null:submit 4][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 5 :: [dc.date.issued:2006-11-16T17.09.52Z][dc.title.null:submit 5][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 6 :: [dc.date.issued:2006-11-16T17.10.18Z][dc.title.null:submit 6][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 7 :: [dc.date.issued:2006-11-16T17.10.43Z][dc.title.null:submit 7][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 8 ::2006- [dc.date.issued:2006-11-16T17.11.08Z][dc.title.null:submit 8][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 9 :: [dc.date.issued:2006-11-16T17.11.30Z][dc.title.null:submit 9][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 10 :: [dc.date.issued:2006-11-16T17.11.56Z][dc.title.null:submit 10][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 11 :: [dc.date.issued:2006-11-16T17.12.19Z][dc.title.null:submit 11][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 12 :: [dc.date.issued:2006-11-16T17.12.45Z][dc.title.null:submit 12][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 13 :: [dc.date.issued:2006-11-16T17.13.09Z][dc.title.null:submit 13][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 14 :: [dc.date.issued:2006-11-16T17.13.31Z][dc.title.null:submit 14][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 15 :: [dc.date.issued:2006-11-16T17.13.52Z][dc.title.null:submit 15][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 16 :: [dc.date.issued:2006-11-16T17.14.16Z][dc.title.null:submit 16][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 17 :: [dc.date.issued:2006-11-16T17.14.37Zsubmit 17 ][dc.title.null:submit 17][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 18 :: [dc.date.issued:2006-11-16T17.14.58Z][dc.title.null:submit 18][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 19 :: [dc.date.issued:2006-11-16T17.15.19Z][dc.title.null:submit 19][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 20 :: [dc.date.issued:2006-11-16T17.15.42Z][dc.title.null:submit 20][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 21 :: [dc.date.issued:2006-11-16T17.16.02Z][dc.title.null:submit 21][dc.contributor.*:Jones,Richard Item ID: 22 : Richard]}} {{Item ID: 22 :: [dc.date.issued:2006-11-16T17.17.24Z][dc.title.null:submit 22][dc.contributor.*:Jones, Richard]}}|| |
TODO: This is currently returning one more row that it's supposed to. This is because I've asked the query to get the row after the current page so that we have a focus for the next page browse. I need to strip this from the result set before giving them to the browse info object
...
- type = dateissued | author | title | subject
- order = ASC | DESC
- value = [free text]
- focus = [item id]
- rpp = [integer: 1 - X]
- sort_by = [integer: 1 - N]
The full browse URL is of the form:
| Panelcode |
|---|
browse?type=<type>&order=<order>&value=<value>&focus=<focus>&rpp=<rpp>&sort_by=<sort_by> |
So for example:
| Panelcode |
|---|
browse?type=author&order=ASC&value=Jones%C2+Richard&focus=34&rpp=10&sort_by=1 |
The following are URLs that have been tested, and the results:
| Panelcode |
|---|
browse?type=dateissued&order=ASC&focus=3&rpp=12&sort_by=0 |
BrowseInfo String Representation: Browsing 2 to 15 of 22||
| Panelcode |
|---|
Listing over 3 columns: dc.date.issued,dc.title.null,dc.contributor.*|| Item {{Item ID: 3 :: [dc.date.issued:2006-11-16T17.09.05Z][dc.title.null:submit 3][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 4 :: [dc.date.issued:2006-11-16T17.09.26Z][dc.title.null:submit4Jones, Richard Item ID: 5 :: 4][dc.contributor.*:Jones, Richard]}} {{Item ID: 5 :: [dc.date.issued:2006-11-16T17.09.52Z][dc.title.null:submit 5][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 6 :: [dc.date.issued:2006-11-16T17.10.18Z][dc.title.null:submit 6][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 7 :: [dc.date.issued:2006-11-16T17.10.43Z][dc.title.null:submit 7][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 8 :: [dc.date.issued:2006-11-16T17.11.08Z][dc.title.null:submit 8][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 9 ::2006 [dc.date.issued:2006-11-16T17.11.30Z][dc.title.null:submit 9][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 10 :: [dc.date.issued:2006-11-16T17.11.56Z][dc.title.null:submit 10][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 11 :: [dc.date.issued:2006-11-16T17.12.19Z][dc.title.null:submit 11][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 12 :: [dc.date.issued:2006-11-16T17.12.45Z][dc.title.null:submit 12][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 13 :: [dc.date.issued:2006-11-16T17.13.09Z][dc.title.null:submit 13][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 14 :: [dc.date.issued:2006-11-16T17.13.31Z][dc.title.null:submit 14][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 15 :: [dc.date.issued:2006-11-16T17.13.52Z][dc.title.null:submit 15][dc.contributor.*:Jones, Richard]}}|| |
(A brief pause then ensued while I added some more information to my debug output so that I can quickly test that things are as they should be; below is the same debug, but with the extra data)
| Panelcode |
|---|
BrowseInfo String Representation: Browsing 2 to 15 of 22 in index: dateissued(data type: date, display type: full|| Listing over 3 columns: dc.date.issued,dc.title.null,dc.contributor.*|| Sorting by: dc.date.issued ASC|| Item {{Item ID: 3 :: [dc.date.issued:2006-11-16T17.09.05Z][dc.title.null:submit 3][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 4 :: [dc.date.issued:2006-11-16T17.09.26Zsubmit 4 ][dc.title.null:submit 4][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 5 :: [dc.date.issued:2006-11-16T17.09.52Z][dc.title.null:submit 5][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 6 :: [dc.date.issued:2006-11-16T17.10.18Z][dc.title.null:submit 6][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 7 :: [dc.date.issued:2006-11-16T17.10.43Z][dc.title.null:submit 7][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 8 :: [dc.date.issued:2006-11-16T17.11.08Z][dc.title.null:submit 8][dc.contributor.*:Jones,Richard Item ID: 9 : Richard]}} {{Item ID: 9 :: [dc.date.issued:2006-11-16T17.11.30Z][dc.title.null:submit 9][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 10 :: [dc.date.issued:2006-11-16T17.11.56Z][dc.title.null:submit 10][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 11 :: [dc.date.issued:2006-11-16T17.12.19Z][dc.title.null:submit 11][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 12 :: [dc.date.issued:2006-11-16T17.12.45Z][dc.title.null:submit 12][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 13 :: [dc.date.issued:2006-11-16T17.13.09Zsubmit 13Jones, Richard Item ID: 14 :: ][dc.title.null:submit 13][dc.contributor.*:Jones, Richard]}} {{Item ID: 14 :: [dc.date.issued:2006-11-16T17.13.31Z][dc.title.null:submit 14][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 15 :: [dc.date.issued:2006-11-16T17.13.52Z][dc.title.null:submit 15][dc.contributor.*:Jones, Richard]}}|| |
Success Metric: type = dateissued, order = ASC (by date issued), focus = 3, results per page = 12 (+1 as documented above), sort by = dateissued (the browse value)
| Panelcode |
|---|
browse?type=dateissued&order=DESC&focus=5&rpp=10&sort_by=1 |
BrowseInfo String Representation: Browsing 22 to 22 of 22 in index: dateissued(data type: date, display type: full||
| Panelcode |
|---|
Listing over 3 columns: dc.date.issued,dc.title.null,dc.contributor.*|| Sorting by: dc.title DESC|||| |
This one evidently has some problems. Going in to find out what's wrong ...
Failure Analysis: This problem appears to be because the value obtained for the focus item (in this case item id 5) is the actual value in the desired index (dateissued), which in this case is "2006-11-16t17:09:52z". The desired value for the focus value should, though, be the value for the focus item id in the relevant sort field of the desired index (in this case it should have been "submit 5").
| Panelcode |
|---|
BrowseInfo String Representation: Browsing 4 to 15 of 22 in index: dateissued(data type: date, display type: full)|| Listing over 3 columns: dc.date.issued,dc.title.null,dc.contributor.*|| Sorting by: dc.title DESC(option 1)|| Item {{Item ID: 5 :: [dc.date.issued:2006-11-16T17.09.52Z][dc.title.null:submit 5][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 4 :: [dc.date.issued:2006-11-16T17.09.26Z][dc.title.null:submit 4][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 3 :: [dc.date.issued:2006-11-16T17.09.05Z][dc.title.null:submit 3][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 22 :: [dc.date.issued:2006-11-16T17.17.24Z][dc.title.null:submit 22][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 21 :: [dc.date.issued:2006-11-16T17.16.02Z][dc.title.null:submit 21][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 20 :: [dc.date.issued:2006-11-16T17.15.42Z][dc.title.null:submit 20][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 2 :: [dc.date.issued:2006-11-16T17.08.42Z][dc.title.null:submit 2][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 19 :: [dc.date.issued:2006-11-16T17.15.19Z][dc.title.null:submit 19][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 18 :: [dc.date.issued:2006-11-16T17.14.58Z][dc.title.null:submit 18][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 17 ::2006 [dc.date.issued:2006-11-16T17.14.37Z][dc.title.null:submit 17][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 16 :: [dc.date.issued:2006-11-16T17.14.16Z][dc.title.null:submit 16][dc.contributor.*:Jones, Richard]}}|| |
Success Metric: type = dateissued, order = DESC (by title), focus = 5, results per page = 10 (+1 as documented above), sort by = title (sort_1)
...
NOTE: it should only be possible to specify the "sort_by" variable from a browse page which is of type "full"
| Panelcode |
|---|
browse?type=author&order=ASC&rpp=25 |
BrowseInfo String Representation: Browsing 0 to 22 of 22 in index: author(data type: text, display type: single)||
| Panelcode |
|---|
Listing over 3 columns: dc.date.issued,dc.title.null,dc.contributor.*|| Sorting by: dc.contributor.* ASC(option 0)|| Item {{Item ID: 22 :: [dc.date.issued:2006-11-16T17.17.24Z][dc.title.null:submit 22][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 21 :: [dc.date.issued:2006-11-16T17.16.02Z][dc.title.null:submit 21][dc.contributor.*:Jones,Richard Item ID: 20 : Richard]}} {{Item ID: 20 :: [dc.date.issued:2006-11-16T17.15.42Z][dc.title.null:submit 20][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 19 :: [dc.date.issued:2006-11-16T17.15.19Z][dc.title.null:submit 19][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 18 :: [dc.date.issued:2006-11-16T17.14.58Z][dc.title.null:submit 18][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 17 :: [dc.date.issued:2006-11-16T17.14.37Z][dc.title.null:submit 17][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 16 :: [dc.date.issued:2006-11-16T17.14.16Zsubmit 16Jones, Richard Item ID: 15 :: ][dc.title.null:submit 16][dc.contributor.*:Jones, Richard]}} {{Item ID: 15 :: [dc.date.issued:2006-11-16T17.13.52Z][dc.title.null:submit 15][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 14 :: [dc.date.issued:2006-11-16T17.13.31Z][dc.title.null:submit 14][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 13 :: [dc.date.issued:2006-11-16T17.13.09Z][dc.title.null:submit 13][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 12 :: [dc.date.issued:2006-11-16T17.12.45Z][dc.title.null:submit 12][dc.contributor.*:Jones,Richard Item ID: 11 : Richard]}} {{Item ID: 11 :: [dc.date.issued:2006-11-16T17.12.19Z][dc.title.null:submit 11][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 10 :: [dc.date.issued:2006-11-16T17.11.56Z][dc.title.null:submit 10][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 9 :: [dc.date.issued:2006-11-16T17.11.30Z][dc.title.null:submit 9][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 8 :: [dc.date.issued:2006-11-16T17.11.08Z][dc.title.null:submit 8][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 7 :: [dc.date.issued:2006-11-16T17.10.43Zsubmit 7Jones, Richard Item ID: 6 :: ][dc.title.null:submit 7][dc.contributor.*:Jones, Richard]}} {{Item ID: 6 :: [dc.date.issued:2006-11-16T17.10.18Z][dc.title.null:submit 6][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 5 :: [dc.date.issued:2006-11-16T17.09.52Z][dc.title.null:submit 5][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 4 :: [dc.date.issued:2006-11-16T17.09.26Z][dc.title.null:submit 4][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 3 :: [dc.date.issued:2006-11-16T17.09.05Z][dc.title.null:submit 3][dc.contributor.*:Jones,Richard Item ID: 2 : Richard]}} {{Item ID: 2 :: [dc.date.issued:2006-11-16T17.08.42Z][dc.title.null:submit 2][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 1 :: [dc.date.issued:2006-11-16T17.08.11Z][dc.title.null:Submit 1][dc.contributor.*:Jones, Richard]}}|| |
Failure Analysis: This is wrong is a fairly significant way, in that since there is only one author name "Jones, Richard", there should only be one result. It therefore needs a DISTINCT clause inserted into the SELECT in the condition that the display type is "single" and there is no value specified for the browse. In order to make testing this index possible I will also add a couple of items by other authors to the test set.
...
I propose, inside the BrowseEngine, to have an initial catch thus:
| Panelcode |
|---|
if (browseIndex.isSingle() && !scope.hasValue()) { \{ browseByValue(scope); } else { \} else \{ browseByItem(scope); }
\}
|
This will then allow us to keep all the logic separate. Obviously much of the supporting methods will work well in both context and can be reused.
...
The SQL that we need to achieve is quite simple, and looks like this
| Panelcode |
|---|
SELECT DISTINCT(value) FROM <index> [WHERE sort_value [<|=] <vfocus>] ORDER BY sort_value<order> LIMIT <rpp> + 1 <order>
LIMIT <rpp> + 1
|
Here we have introduced one new variable (which has been fed back to the earlier list of UI variables) called "vfocus", and which is the text value of the target focus. For pagination this is obtained by the +1 on the next page in the LIMIT portion of the query. More on this later.
...
- The SQL appears to be coming together quite quickly
- We have a method called getFocusValue which is supposed to return a string value from the item id integer. It would be good if we can generalise this so that it dealt with both "focus" and "vfocus". I've added a vFocus member variable to the BrowserScope object. The next thing to do (probably not until Friday now) is track this back through to the Servlet to ensure that it gets properly populated by the UI
- The UI URL parameter "vfocus" has been added (and retrofitted to the list above)
- the vfocus parameter has been propagated from the URL to the browse engine, and the engine has been modified so that it should now be able to deal with both browsing by item or by value. Once this compiles, testing info to follow...
- Having forgotten how to write SELECT DISTINCT statements properly, we modify our SQL for value browsing to be:
| Panelcode |
|---|
SELECT DISTINCT(value), sort_value FROM index_<n> [WHERE sort_value [<|=] <vfocus>] ORDER BY sort_value <order> LIMIT <rpp> + 1 |
- in order to make the BrowseInfo.toString method work with the new value browse, I need to go in and make some changes
...
Having got through end-to-end on the first single value browse (author), we are ready to throw some more stuff at the browse engine and see how it copes. Let's start simple:
| Panelcode |
|---|
browse?type=author |
BrowseInfo String Representation: Browsing 0 to 1 of 22 in index: author(data type: text, display type: single)||
| Panelcode |
|---|
Listing over 3 columns: dc.date.issued,dc.title.null,dc.contributor.*|| Sorting by: dc.contributor.* ASC(option 0)|| { { \{ \{ Value: Jones, Richard\}\}|| |
Failure Analysis: while this looks correct, we can see that the count value 0 - 1 of 22 is incorrect. This is because of the following factors: the BrowseEngine.getTotalResults is not implementing DISTINCT, and because of the way that BrowseInfo.toString makes a minor error in calculating the range (although the actual range is correct). It is also displaying which columns it lists over, even though it will not do this. These to be fixed before the next test.
| Panelcode |
|---|
BrowseInfo String Representation: Browsing 1 to 1 of 1 in index: author(data type: text, display type: single)|| Listing single column: dc.contributor.*|| Sorting by: dc.contributor.* ASC(option 0)|| { { \{ \{ Value: Jones, Richard\}\}|| |
Success Metric: type = author, 1 result
...
Testing the Browse URL (part 3)
| Panelcode |
|---|
browse?type=author |
BrowseInfo String Representation: Browsing 1 to 6 of 6 in index: author(data type: text, display type: single)||
| Panelcode |
|---|
Listing single column: dc.contributor.*|| Sorting by: dc.contributor.* ASC(option 0)|| { \{ \{Value: Ardman, Alfred\}} { \} \{ \{Value: Boothroyd, Betty\}} { \} \{ \{Value: Chaplin, Charlie\}} { \} \{ \{Value: Decimal, Dewey\}} { \} \{ \{Value: Eagle, Eddie\}} { \} \{ \{Value: Jones, Richard\}\}|| |
Success Metric: single value browse, type-author, results=6, sort by name ASC
...
Note: I have gone back and re-run all the tests done above and they appear, withough significant analysis, to be correct. I have also re-run the last URL which caused the problems before, and it produces exactly the same results as above, which is what we would expect.
| Panelcode |
|---|
browse?type=author&order=DESC&rpp=3&vfocus=Boothroyd%2C+Betty |
BrowseInfo String Representation: Browsing 26 to 27 of 6 in index: author(data type: text, display type: single)||
| Panelcode |
|---|
Listing single column: dc.contributor.*|| Sorting by: dc.contributor.* DESC(option 0)|| { \{ \{Value: Boothroyd, Betty\}} { \} \{ \{Value: Ardman, Alfred\}\}|| |
Failure Analysis: sigh. Well this has behaved pretty much correctly, in so much as it starts with "Boothroyd, Betty" and works its way in descending order to the end of the list. Unfortunately, the "26 to 27" bit is a little out! This appears to be because the count prior to the current value is not employing a DISTINCT(value) section to the query. Therefore, the range is correct, and if the starting point were correct then everything would have gone as planned.
| Panelcode |
|---|
BrowseInfo String Representation: Browsing 5 to 6 of 6 in index: author(data type: text, display type: single)|| Listing single column: dc.contributor.*|| Sorting by: dc.contributor.* DESC(option 0)|| { \{ \{Value: Boothroyd, Betty\}} { \} \{ \{Value: Ardman, Alfred\}\}|| |
Success Metric: type=author, value focus = Boothroyd, Betty, sorted by value descending.
| Panelcode |
|---|
browse?type=title&order=ASC&focus=13&rpp=5&sort_by=2 |
BrowseInfo String Representation: Browsing 13 to 18 of 27 in index: title(data type: title, display type: full)||
| Panelcode |
|---|
Listing over 3 columns: dc.date.issued,dc.title.null,dc.contributor.*|| Sorting by: dc.date.issued ASC(option 2)|| Item {{Item ID: 13 :: [dc.date.issued:2006-11-16T17.13.09Z][dc.title.null:submit 13][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 14 :: [dc.date.issued:2006-11-16T17.13.31Z][dc.title.null:submit 14][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 15 :: [dc.date.issued:2006-11-16T17.13.52Z][dc.title.null:submit 15][dc.contributor.*:Jones,Richard Item ID: 16 : Richard]}} {{Item ID: 16 :: [dc.date.issued:2006-11-16T17.14.16Z][dc.title.null:submit 16][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 17 :: [dc.date.issued:2006-11-16T17.14.37Z][dc.title.null:submit 17][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 18 :: [dc.date.issued:2006-11-16T17.14.58Z][dc.title.null:submit 18][dc.contributor.*:Jones, Richard]}}|| |
Success Metric: type=title, order = undeterminable for this set, as all the authors are the same, focus = 13, range is correct (+1 as above), rpp = 5 (+1). Sorting by author needs to be bourne out by other tests
| Panelcode |
|---|
browse?type=title&order=DESC&rpp=30&sort_by=2 |
BrowseInfo String Representation: Browsing 1 to 27 of 27 in index: title(data type: title, display type: full)||
| Panelcode |
|---|
Listing over 3 columns: dc.date.issued,dc.title.null,dc.contributor.*|| Sorting by: dc.date.issued DESC(option 2)|| Item {{Item ID: 27 :: [dc.date.issued:2006-11-30T12.56.42Z][dc.title.null:Submit E][dc.contributor.*:Eagle,Eddie Item Eddie]}} {{Item ID: 26 :: [dc.date.issued:2006-11-30T12.56.19Z][dc.title.null:Submit D][dc.contributor.*:Decimal,Dewey Item Dewey]}} {{Item ID: 25 :: [dc.date.issued:2006-11-30T12.55.58Z][dc.title.null:Submit C][dc.contributor.*:Chaplin,Charlie Item Charlie]}} {{Item ID: 24 :: [dc.date.issued:2006-11-30T12.55.33Z][dc.title.null:Submit B][dc.contributor.*:Boothroyd,Betty Item Betty]}} {{Item ID: 23 :: [dc.date.issued:2006-11-30T12.49.29Z][dc.title.null:Submit A][dc.contributor.*:Ardman,Alfred Item Alfred]}} {{Item ID: 22 ::2006- [dc.date.issued:2006-11-16T17.17.24Z][dc.title.null:submit 22][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 21 :: [dc.date.issued:2006-11-16T17.16.02Z][dc.title.null:submit 21][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 20 :: [dc.date.issued:2006-11-16T17.15.42Z][dc.title.null:submit 20][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 19 :: [dc.date.issued:2006-11-16T17.15.19Z][dc.title.null:submit 19][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 18 :: [dc.date.issued:2006-11-16T17.14.58Z][dc.title.null:submit 18][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 17 :: [dc.date.issued:2006-11-16T17.14.37Z][dc.title.null:submit 17][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 16 :: [dc.date.issued:2006-11-16T17.14.16Z][dc.title.null:submit 16][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 15 :: [dc.date.issued:2006-11-16T17.13.52Z][dc.title.null:submit 15][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 14 :: [dc.date.issued:2006-11-16T17.13.31Z][dc.title.null:submit 14][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 13 :: [dc.date.issued:2006-11-16T17.13.09Zsubmit 13 ][dc.title.null:submit 13][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 12 :: [dc.date.issued:2006-11-16T17.12.45Z][dc.title.null:submit 12][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 11 :: [dc.date.issued:2006-11-16T17.12.19Z][dc.title.null:submit 11][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 10 :: [dc.date.issued:2006-11-16T17.11.56Z][dc.title.null:submit 10][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 9 :: [dc.date.issued:2006-11-16T17.11.30Z][dc.title.null:submit 9][dc.contributor.*:Jones,Richard Item ID: 8 : Richard]}} {{Item ID: 8 :: [dc.date.issued:2006-11-16T17.11.08Z][dc.title.null:submit 8][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 7 :: [dc.date.issued:2006-11-16T17.10.43Z][dc.title.null:submit 7][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 6 :: [dc.date.issued:2006-11-16T17.10.18Z][dc.title.null:submit 6][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 5 :: [dc.date.issued:2006-11-16T17.09.52Z][dc.title.null:submit 5][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 4 :: [dc.date.issued:2006-11-16T17.09.26Zsubmit 4Jones, Richard Item ID: 3 :: ][dc.title.null:submit 4][dc.contributor.*:Jones, Richard]}} {{Item ID: 3 :: [dc.date.issued:2006-11-16T17.09.05Z][dc.title.null:submit 3][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 2 :: [dc.date.issued:2006-11-16T17.08.42Z][dc.title.null:submit 2][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 1 :: [dc.date.issued:2006-11-16T17.08.11Z][dc.title.null:Submit 1][dc.contributor.*:Jones, Richard]}}|| |
Success Metric: type=title, order = date issued descending (option 2 was date issued not author, as I have not (and cannot) set an author sort option (see notes above)). less than 30 results are on the page, and this is all of them
| Panelcode |
|---|
browse?type=subject&order=DESC&rpp=10 |
BrowseInfo String Representation: Browsing 1 to 2 of 2 in index: subject(data type: text, display type: single)||
| Panelcode |
|---|
Listing single column: dc.subject.*|| Sorting by: dc.subject.* DESC(option 0)|| { \{ \{Value: fdsa\}} { \} \{ \{Value: asdf\}\}|| |
Success Metric: type=subject, order = subject descending, all values in results
| Panelcode |
|---|
browse?type=subject&order=ASC&rpp=10&vfocus=fdsa |
BrowseInfo String Representation: Browsing 2 to 2 of 2 in index: subject(data type: text, display type: single)||
| Panelcode |
|---|
Listing single column: dc.subject.*|| Sorting by: dc.subject.* ASC(option 0)|| { \{ \{Value: fdsa\}\}|| |
Success Metric: type=subject, order = subject ascending (even though there is only one result, because we focus on "fdsa" we know that it is sorting the right way), vfocus = fdsa (asdf would be number 1, which is not displayed)
Now we can continue to test the "second level browse" pages, which are Single/Value browses which have value parameters specified. This could be the source of some more bugs
| Panelcode |
|---|
browse?type=author&order=ASC&value=Jones%2C+Richard&rpp=10&sort_by=1 |
Failure Analysis: this URL returns a blank page. This appears to be caused by a problem with turning the BrowseInfo object into a String
| Panelcode |
|---|
java.lang.ClassCastException: org.dspace.browse.BrowseItem at at org.dspace.browse.BrowseInfo.valueListingString(BrowseInfo.java:462) at at org.dspace.browse.BrowseInfo.toString(BrowseInfo.java:336) |
This is because there is a test BrowseIndex.isSingle which only remarks on the browse type - it does not consider whether we are at the top or second level of the browse. The BrowseInfo object needs to know whether it is doing top level or second level browsing, as does the BrowseScope object for other uses. I propose the addition of isTopLevel and isSecondLevel to both of these objects, and to have them populated by the BrowseServlet
...
- This means that things which were done previously in the BrowseEngine thus:
| Panelcode |
|---|
if (browseIndex.isSingle() && scope.hasValue()) |
can now be done thus:
| Panelcode |
|---|
if (scope.isSecondLevel()) |
Testing the Browse URL (part 4)
| Panelcode |
|---|
browse?type=author&order=ASC&value=Jones%2C+Richard&rpp=10&sort_by=1 |
BrowseInfo String Representation: Browsing 1 to 11 of 22 in index: author(data type: text, display type: single)||
| Panelcode |
|---|
Listing single column: dc.contributor.*|| Sorting by: dc.title ASC(option 1)|| Item {{Item ID: 1 :: [dc.date.issued:2006-11-16T17.08.11Z][dc.title.null:Submit 1][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 10 :: [dc.date.issued:2006-11-16T17.11.56Z][dc.title.null:submit 10][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 11 :: [dc.date.issued:2006-11-16T17.12.19Z][dc.title.null:submit 11][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 12 :: [dc.date.issued:2006-11-16T17.12.45Z][dc.title.null:submit 12][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 13 :: [dc.date.issued:2006-11-16T17.13.09Z][dc.title.null:submit 13][dc.contributor.*:Jones,Richard Item ID: 14 : Richard]}} {{Item ID: 14 :: [dc.date.issued:2006-11-16T17.13.31Z][dc.title.null:submit 14][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 15 :: [dc.date.issued:2006-11-16T17.13.52Z][dc.title.null:submit 15][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 16 :: [dc.date.issued:2006-11-16T17.14.16Z][dc.title.null:submit 16][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 17 :: [dc.date.issued:2006-11-16T17.14.37Z][dc.title.null:submit 17][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 18 :: [dc.date.issued:2006-11-16T17.14.58Zsubmit 18Jones, Richard Item ID: 19 :: ][dc.title.null:submit 18][dc.contributor.*:Jones, Richard]}} {{Item ID: 19 :: [dc.date.issued:2006-11-16T17.15.19Z][dc.title.null:submit 19][dc.contributor.*:Jones, Richard]}}|| |
Success Metric: somewhat miraculous that there were no significant problems type=author, level=2, sort by = title, ASC, value="Jones, Richard"
| Panelcode |
|---|
browse?type=author&order=DESC&value=Jones%2C+Richard&rpp=5&sort_by=1&vfocus=submit+13 |
BrowseInfo String Representation: Browsing 1 to 6 of 22 in index: author(data type: text, display type: single)||
| Panelcode |
|---|
Listing single column: dc.contributor.*|| Sorting by: dc.title DESC(option 1)|| Item {{Item ID: 9 :: [dc.date.issued:2006-11-16T17.11.30Z][dc.title.null:submit 9][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 8 :: [dc.date.issued:2006-11-16T17.11.08Z][dc.title.null:submit 8][dc.contributor.*:Jones,Richard Item ID: 7 : Richard]}} {{Item ID: 7 :: [dc.date.issued:2006-11-16T17.10.43Z][dc.title.null:submit 7][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 6 :: [dc.date.issued:2006-11-16T17.10.18Z][dc.title.null:submit 6][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 5 :: [dc.date.issued:2006-11-16T17.09.52Z][dc.title.null:submit 5][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 4 :: [dc.date.issued:2006-11-16T17.09.26Z][dc.title.null:submit 4][dc.contributor.*:Jones, Richard]}}|| |
Failure Analysis: it appears that this browse has not used the "vfocus" variable in the SQL. This means that the SQL has not been written quite correctly. All other features of the browse have functioned correctly as far as I can tell.
...
Testing the Browse URL (part 5)
| Panelcode |
|---|
browse?type=author&order=DESC&value=Jones%2C+Richard&rpp=5&sort_by=1&vfocus=submit+13 |
BrowseInfo String Representation: Browsing 23 to 27 of 22 in index: author(data type: text, display type: single)||
| Panelcode |
|---|
Listing single column: dc.contributor.*|| Sorting by: dc.title DESC(option 1)|| Item {{Item ID: 13 :: [dc.date.issued:2006-11-16T17.13.09Z][dc.title.null:submit 13][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 12 :: [dc.date.issued:2006-11-16T17.12.45Z][dc.title.null:submit 12][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 11 :: [dc.date.issued:2006-11-16T17.12.19Z][dc.title.null:submit 11][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 10 :: [dc.date.issued:2006-11-16T17.11.56Z][dc.title.null:submit 10][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 1 ::2006 [dc.date.issued:2006-11-16T17.08.11Z][dc.title.null:Submit 1][dc.contributor.*:Jones, Richard]}}|| |
Failure Analysis: well, it appears to have selected the correct values, and the reason there aren't 6 (rpp + 1) is because it reached the end of the index for those parameters. They are in the correct order for the correct value, but now we are in the range 23 - 27 of 22, which is interesting. The range is correct, so only the start value is at fault. This looks like an application (or misapplication) of the DISTINCT SQL construct.
| Panelcode |
|---|
BrowseInfo String Representation: Browsing 7 to 11 of 22 in index: author(data type: text, display type: single)|| Listing single column: dc.contributor.*|| Sorting by: dc.title DESC(option 1)|| Item {{Item ID: 13 :: [dc.date.issued:2006-11-16T17.13.09Z][dc.title.null:submit 13][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 12 :: [dc.date.issued:2006-11-16T17.12.45Z][dc.title.null:submit 12][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 11 :: [dc.date.issued:2006-11-16T17.12.19Z][dc.title.null:submit11Jones, Richard Item ID: 10 :: 11][dc.contributor.*:Jones, Richard]}} {{Item ID: 10 :: [dc.date.issued:2006-11-16T17.11.56Z][dc.title.null:submit 10][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 1 :: [dc.date.issued:2006-11-16T17.08.11Z][dc.title.null:Submit 1][dc.contributor.*:Jones, Richard]}}|| |
Success Metric: good news type=author, order = title descending (sorting by title), value = "Richard, Jones" (and it is known that at this point there are 22 items by that author), vfocus = "submit 13"
| Panelcode |
|---|
browse?type=author&order=ASC&value=Eagle%2C+Eddie&rpp=10&sort_by=2 |
BrowseInfo String Representation: Browsing 1 to 1 of 1 in index: author(data type: text, display type: single)||
| Panelcode |
|---|
Listing single column: dc.contributor.*|| Sorting by: dc.date.issued ASC(option 2)|| Item {{Item ID: 27 :: [dc.date.issued:2006-11-30T12.56.42Z][dc.title.null:Submit E][dc.contributor.*:Eagle, Eddie]}}|| |
Success Metric: type=author, order = unknowable, but should be by date, value ="Eagle, Eddie" (and it is known that there is only 1 item by this author)
| Panelcode |
|---|
browse?type=subject&order=DESC&value=asdf&rpp=10&sort_by=1 |
BrowseInfo String Representation: Browsing 1 to 11 of 22 in index: subject(data type: text, display type: single)||
| Panelcode |
|---|
Listing single column: dc.subject.*|| Sorting by: dc.title DESC(option 1)|| Item {{Item ID: 9 :: [dc.date.issued:2006-11-16T17.11.30Z][dc.title.null:submit 9][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 8 :: [dc.date.issued:2006-11-16T17.11.08Z][dc.title.null:submit 8][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 7 :: [dc.date.issued:2006-11-16T17.10.43Z][dc.title.null:submit 7][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 6 :: [dc.date.issued:2006-11-16T17.10.18Z][dc.title.null:submit 6][dc.contributor.*:Jones,Richard Item ID: 5 : Richard]}} {{Item ID: 5 :: [dc.date.issued:2006-11-16T17.09.52Z][dc.title.null:submit 5][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 4 :: [dc.date.issued:2006-11-16T17.09.26Z][dc.title.null:submit 4][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 3 :: [dc.date.issued:2006-11-16T17.09.05Z][dc.title.null:submit 3][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 22 :: [dc.date.issued:2006-11-16T17.17.24Z][dc.title.null:submit 22][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 21 :: [dc.date.issued:2006-11-16T17.16.02Zsubmit 21Jones, Richard Item ID: 20 :: ][dc.title.null:submit 21][dc.contributor.*:Jones, Richard]}} {{Item ID: 20 :: [dc.date.issued:2006-11-16T17.15.42Z][dc.title.null:submit 20][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 2 :: [dc.date.issued:2006-11-16T17.08.42Z][dc.title.null:submit 2][dc.contributor.*:Jones, Richard]}}|| |
TODO: the results don't yet tell us which value we are browsing on - this will need to be fixed for the real UI
...
Browse URL Testing Summary
border="1"
| Panelcode | ||||||||
|---|---|---|---|---|---|---|---|---|
A summary of the URL parameters tested above- ! test |- ! test !! mode !! type !! order !! value !! focus !! vfocus !! rpp !! sort_by | ||||||||
- | 1 | dateissued | ASC | 3 | 12 | 0 | ||
- | ||||||||
2 | Full/Item | dateissued | DESC | N/A | 5 | N/A | 10 | 1 |
- | 3 | Single/Value | author | ASC | N/A | 25 | N/A | |
- | 4 | Single/Value | author | DESC | N/A | Boothroyd, Betty | 3 | N/A |
- | ||||||||
5 | Full/Item | title | ASC | N/A | 13 | N/A | 5 | 2 |
- | 6 | Full/Item | title | DESC | N/A | N/A | 30 | 2 |
- | 7 | Single/Value | subject | DESC | N/A | 10 | N/A | |
- | 8 | Single/Value | subject | ASC | N/A | fdsa | 10 | N/A |
- | 9 | Single/Value with Value | author | ASC | Jones, Richard | N/A | 10 | 1 |
- | ||||||||
10 | Single/Value with Value | author | DESC | Jones, Richard | N/A | submit 13 | 5 | 1 |
- | 11 | Single/Value with Value | author | ASC | Eagle, Eddie | N/A | 10 | 2 |
- | 12 | Single/Value with Value | subject | DESC | asdf | N/A | 10 | 1
Progress Update: 30-11-2006, 16:50 GMT
...
The "starts_with" parameter is used through the user interface to indicate what value the search should look for strings starting with to display. For example:
| Panelcode |
|---|
starts_with=Jon |
should match "Jones, Richard". This also works with dates, where
| Panelcode |
|---|
year=2006&month=01 |
should be equivalent to:
| Panelcode |
|---|
starts_with=2006-01 |
and therefore will match all things published in the year 2006 in January
Converting "year" and "month" into "starts_with" will be done in the BrowseServlet. The impact that it will have on the SQL is to require us to stop using "= 'some value'" and start using "LIKE 'some value%'" when a "starts_with" parameter is available. So the SQL will look like this:
| Panelcode |
|---|
SELECT * FROM <index> WHERE sort_value [[<|=] [<value> | _focus-value_] | LIKE <starts_with>%] AND ] [AND collection_id = _collection_ AND ] [AND community_id = _community_ ORDER BY <sortBy> <order> LIMIT <resultsperpage + 1> ]
ORDER BY <sortBy> <order>
LIMIT <resultsperpage + 1>
|
Note: this utilises the database's regular expression engine, which will have a performance impact. Therefore, we cannot always use LIKE for convenience, we must only use it when there is a "starts_with" parameter.
...
Testing the Browse URL (part 6)
| Panelcode |
|---|
browse?type=dateissued&order=ASC&rpp=10&sort_by=0&starts_with=2006-11 |
BrowseInfo String Representation: Browsing 1 to 11 of 27 in index: dateissued(data type: date, display type: full)||
| Panelcode |
|---|
Listing over 3 columns: dc.date.issued,dc.title.null,dc.contributor.* starting with value: 2006-11|| Sorting by: dc.date.issued ASC(option 0)|| Item {{Item ID: 1 :: [dc.date.issued:2006-11-16T17.08.11Z][dc.title.null:Submit 1][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 2 :: [dc.date.issued:2006-11-16T17.08.42Z][dc.title.null:submit 2][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 3 :: [dc.date.issued:2006-11-16T17.09.05Z][dc.title.null:submit 3][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 4 :: [dc.date.issued:2006-11-16T17.09.26Z][dc.title.null:submit 4][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 5 :: [dc.date.issued:2006-11-16T17.09.52Z][dc.title.null:submit 5][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 6 :: [dc.date.issued:2006-11-16T17.10.18Z][dc.title.null:submit 6][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 7 :: [dc.date.issued:2006-11-16T17.10.43Z][dc.title.null:submit 7][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 8 :: [dc.date.issued:2006-11-16T17.11.08Z][dc.title.null:submit 8][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 9 :: [dc.date.issued:2006-11-16T17.11.30Z][dc.title.null:submit 9][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 10 :: [dc.date.issued:2006-11-16T17.11.56Z][dc.title.null:submit 10][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 11 :: [dc.date.issued:2006-11-16T17.12.19Z][dc.title.null:submit 11][dc.contributor.*:Jones, Richard]}}|| |
Success Metric: type=dateissued, order = date issued ascending, rpp= 10 (+1), starts with "2006-11"; we know that there are 27 items in the database, and they were all entered in November 2006
| Panelcode |
|---|
browse?type=dateissued&order=DESC&rpp=10&sort_by=0&starts_with=2006-11-30 |
BrowseInfo String Representation: Browsing 6 to 10 of 27 in index: dateissued(data type: date, display type: full)||
| Panelcode |
|---|
Listing over 3 columns: dc.date.issued,dc.title.null,dc.contributor.* starting with value: 2006-11-30|| Sorting by: dc.date.issued DESC(option 0)|| Item {{Item ID: 27 :: [dc.date.issued:2006-11-30T12.56.42Z][dc.title.null:Submit E][dc.contributor.*:Eagle,Eddie Item Eddie]}} {{Item ID: 26 :: [dc.date.issued:2006-11-30T12.56.19Z][dc.title.null:Submit D][dc.contributor.*:Decimal,Dewey Item Dewey]}} {{Item ID: 25 :: [dc.date.issued:2006-11-30T12.55.58Z][dc.title.null:Submit C][dc.contributor.*:Chaplin,Charlie Item ID: 24 : Charlie]}} {{Item ID: 24 :: [dc.date.issued:2006-11-30T12.55.33Z][dc.title.null:Submit B][dc.contributor.*:Boothroyd,Betty Item Betty]}} {{Item ID: 23 :: [dc.date.issued:2006-11-30T12.49.29Z][dc.title.null:Submit A][dc.contributor.*:Ardman, Alfred]}}|| |
Failure Analysis: the perplexing thing about this is that only the results that actually start with the value are supplied. A moment examining the SQL we wrote shows us that we were over-zealous in our logic. There is no need to invoke the regular expression engine, it is simply enough to supply the "starts_with" parameter in place of the "focus" or "vfocus" in the query, thus:
| Panelcode |
|---|
SELECT * FROM <index> WHERE sort_value [<<value> AND |=] [<value> | _starts_with_] [AND collection_id = _collection_ AND ] [AND community_id = _community_ ORDER BY <sortBy> <order> LIMIT <resultsperpage + 1> ]
ORDER BY <sortBy> <order>
LIMIT <resultsperpage + 1>
|
The code update to achieve this now gives us this result:
| Panelcode |
|---|
BrowseInfo String Representation: Browsing 6 to 16 of 27 in index: dateissued(data type: date, display type: full)|| Listing over 3 Listing over 3 columns: dc.date.issued,dc.title.null,dc.contributor.* starting with value: 2006-11-30|| Sorting by: dc.date.issued DESC(option 0)|| Item {{Item ID: 22 :: [dc.date.issued:2006-11-16T17.17.24Z][dc.title.null:submit 22][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 21 :: [dc.date.issued:2006-11-16T17.16.02Z][dc.title.null:submit 21][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 20 :: [dc.date.issued:2006-11-16T17.15.42Z][dc.title.null:submit 20][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 19 :: [dc.date.issued:2006-11-16T17.15.19Z][dc.title.null:submit 19][dc.contributor.*:Jones,Richard Item ID: 18 : Richard]}} {{Item ID: 18 :: [dc.date.issued:2006-11-16T17.14.58Z][dc.title.null:submit 18][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 17 :: [dc.date.issued:2006-11-16T17.14.37Z][dc.title.null:submit 17][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 16 :: [dc.date.issued:2006-11-16T17.14.16Z][dc.title.null:submit 16][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 15 :: [dc.date.issued:2006-11-16T17.13.52Z][dc.title.null:submit 15][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 14 :: [dc.date.issued:2006-11-16T17.13.31Zsubmit 14Jones, Richard Item ID: 13 :: ][dc.title.null:submit 14][dc.contributor.*:Jones, Richard]}} {{Item ID: 13 :: [dc.date.issued:2006-11-16T17.13.09Z][dc.title.null:submit 13][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 12 :: [dc.date.issued:2006-11-16T17.12.45Z][dc.title.null:submit 12][dc.contributor.*:Jones, Richard]}}|| |
Failure Analysis: this is not as wrong as it looks. It has been produced as an oddity in the way that string comparisons are dealt with in the database. The query which generated this looks like this:
| Panelcode |
|---|
SELECT * FROM index_1 WHERE sort_value <= '2006-11-30' ORDER BY sort_value DESC LIMIT 11 |
which will actually only match everything after 2006-11-30, not including. The original Browse code notes this problem as follows:
| Code Block |
|---|
/*
* When the user is browsing with the most recent items first,
* the browse code algorithm doesn't quite do what some people
* might expect. For example, if in the index there are entries:
*
* Mar-2000 15-Feb-2000 6-Feb-2000 15-Jan-2000
*
* and the user has selected "Feb 2000" as the start point for
* the browse, the browse algorithm will start at the first
* point in that index *after* "Feb 2000". "Feb 2000" would
* appear in the index above between 6-Feb-2000 and 15-Jan-2000.
* So, the browse code in this case will start the browse at
* |
| Panel |
/* "15-Jan-2000". This isn't really what users are likelytowant: to * want: They're more likely to want the browse to start atthefirst Feb 2000 date, the * first Feb 2000 date, i.e. 15-Feb-2000. A similarscenario */ scenario
* occurs when the user enters just a year. Our quick hack to
* produce this behaviour is to add "-32" to the startsWith
* variable, when sorting with most recent items first. This
* means the browse code starts at the topmost item in the index
* that matches the user's input, rather than the point in the
* index where the user's input would appear.
*/
|
We will adopt the same approach for the new browse code. This means that we must abandon this particular test (see note below).
Note: there is an implied limit to the functionality of the Browse Engine here. All string comparison problems for dates are overcome by the application of "-32" to the end of the string. This works perfectly well for years without months and months without days (because there are less than 32 in both), but does not work at the days level (that is, if starts_with=2006-11-30, then the query will be on a date before "2006-11-30-32", which does not successfully compare to a date of the form "2006-11-30T17:17:24Z"
| Panelcode |
|---|
browse?type=author&order=ASC&rpp=10&sort_by=0&starts_with=Jon |
BrowseInfo String Representation: Browsing 6 to 6 of 6 in index: author(data type: text, display type: single)||
| Panelcode |
|---|
Listing single column: dc.contributor.* starting with value: Jon|| Sorting by: dc.contributor.* ASC(option 0)|| { \{ \{Value: Jones, Richard\}\}|| |
Success Metric: type=author, sort by = author ascending (otherwise, Jones, Richard wouldn't be last). Correct range and everything.
| Panelcode |
|---|
browse?type=title&order=DESC&rpp=10&sort_by=0&starts_with=submit+2 |
BrowseInfo String Representation: Browsing 16 to 26 of 27 in index: title(data type: title, display type: full)||
| Panelcode |
|---|
Listing over 3 columns: dc.date.issued,dc.title.null,dc.contributor.* starting with value: submit 2|| Sorting by: dc.title DESC(option 0)|| Item {{Item ID: 2 :: [dc.date.issued:2006-11-16T17.08.42Z][dc.title.null:submit 2][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 19 :: [dc.date.issued:2006-11-16T17.15.19Z][dc.title.null:submit19Jones, Richard Item ID: 18 :: 19][dc.contributor.*:Jones, Richard]}} {{Item ID: 18 :: [dc.date.issued:2006-11-16T17.14.58Z][dc.title.null:submit 18][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 17 :: [dc.date.issued:2006-11-16T17.14.37Z][dc.title.null:submit 17][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 16 :: [dc.date.issued:2006-11-16T17.14.16Z][dc.title.null:submit 16][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 15 :: [dc.date.issued:2006-11-16T17.13.52Z][dc.title.null:submit 15][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 14 ::2006 [dc.date.issued:2006-11-16T17.13.31Z][dc.title.null:submit 14][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 13 :: [dc.date.issued:2006-11-16T17.13.09Z][dc.title.null:submit 13][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 12 :: [dc.date.issued:2006-11-16T17.12.45Z][dc.title.null:submit 12][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 11 :: [dc.date.issued:2006-11-16T17.12.19Z][dc.title.null:submit 11][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 10 :: [dc.date.issued:2006-11-16T17.11.56Z][dc.title.null:submit 10][dc.contributor.*:Jones, Richard]}}|| |
Success Metric: type=title, sort by = title descending, staring with the last submit+2 value when descending
Note: using "starts_with" with an order=DESC is an odd thing to do, and produces the above technically correct, but slightly misleading result. Worth mentioning.
| Panelcode |
|---|
browse?type=subject&order=ASC&rpp=10&sort_by=0&starts_with=a |
BrowseInfo String Representation: Browsing 1 to 2 of 2 in index: subject(data type: text, display type: single)||
| Panelcode |
|---|
Listing single column: dc.subject.* starting with value: a|| Sorting by: dc.subject.* ASC(option 0)|| { \{ \{Value: asdf\}} { \} \{ \{Value: fdsa\}\}|| |
Success Metric: type=subject, sort by = subject ascending, starting with a
| Panelcode |
|---|
browse?type=author&order=ASC&rpp=10&sort_by=1&starts_with=submit+2&value=Jones%2C+Richard |
BrowseInfo String Representation: Browsing 2 to 12 of 22 in index: author(data type: text, display type: single)||
| Panelcode |
|---|
Listing single column: dc.contributor.* starting with value: submit 2|| Sorting by: dc.title ASC(option 1)|| Item {{Item ID: 2 :: [dc.date.issued:2006-11-16T17.08.42Z][dc.title.null:submit 2][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 20 :: [dc.date.issued:2006-11-16T17.15.42Z][dc.title.null:submit20Jones, Richard Item ID: 21 :: 20][dc.contributor.*:Jones, Richard]}} {{Item ID: 21 :: [dc.date.issued:2006-11-16T17.16.02Z][dc.title.null:submit 21][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 22 :: [dc.date.issued:2006-11-16T17.17.24Z][dc.title.null:submit 22][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 3 :: [dc.date.issued:2006-11-16T17.09.05Z][dc.title.null:submit 3][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 4 :: [dc.date.issued:2006-11-16T17.09.26Z][dc.title.null:submit 4][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 5 ::2006- [dc.date.issued:2006-11-16T17.09.52Z][dc.title.null:submit 5][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 6 :: [dc.date.issued:2006-11-16T17.10.18Z][dc.title.null:submit 6][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 7 :: [dc.date.issued:2006-11-16T17.10.43Z][dc.title.null:submit 7][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 8 :: [dc.date.issued:2006-11-16T17.11.08Z][dc.title.null:submit 8][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 9 :: [dc.date.issued:2006-11-16T17.11.30Z][dc.title.null:submit 9][dc.contributor.*:Jones, Richard]}}|| |
Success Metric: type=author value = "jones, richard", sort by = title, starting with submit 2, and going up
...
Testing Browse URL Summary (part 2)
border="1"
| Panelcode |
|---|
A summary of the URL parameters tested above- ! test |- ! test !! mode !! type !! order !! value !! focus !! vfocus !! rpp !! sort_by !! starts_with |
- | 13 | dateissued | ASC | 10 | 0 | 2006-11 |
- | 14 | Single/Value | author | ASC | 10 | 0 | Jon |
- | 15 | Full/Item | title | DESC | 10 | 0 | submit 2 |
- | 16 | Single/Value | subject | ASC | 10 | 0 | a |
- | 17 | Value | author | ASC | 10 | 1 | submit 2
Introducing restriction to Community or Collection
Community and Collection data can be obtained from the URL, when the browse URL is of the form:
| Panelcode |
|---|
handle/123456789/4321/browse?.... |
Where 123456789/4321 is the handle of the community or collection to be browsed in.
When we are inside a community or collection the browse must be done on one of the views created on the data which lists browse results by collection id. This means that we can construct SQL queries of the form:
| Panelcode |
|---|
SELECT * FROM [community|collection]_<index> WHERE WHERE [collection_id|community_id] =<collection> AND [<collection>|_community_] AND sort_value [[<<value> ORDER BY <sortBy> <order> LIMIT <rpp> + 1 |=] [<value> | _starts_with_]
ORDER BY <sortBy> <order>
LIMIT <rpp> + 1
|
To achieve this we need to place the community or collection object into the BrowseScope to go into the BrowseEngine. The engine can then construct the query in the same way for both item and value browses, for example:
| Panelcode |
|---|
if (scope.isCollection()) { table = \{ table = browseIndex.getTableName(false, true); } else if \} else if (scope.isCommunity()) { \{ .... |
To obtain the table name, and similarly to construct the relevant segment of the WHERE clause. The community or collectiont then needs to be passed back into the BrowseInfo object so that it can report on the scope of the browse.
...
- At home time, you had implemented the Servlet end of things for taking the collection or community, and added them to the BrowseScope object, with the relevant accessors. Next, implement in the BrowseEngine.
- Looks like I've been excessive with my application of the constraints to community and collection. Probably this is just a missing check in the BrowseEngine, no biggie:
| Panelcode |
|---|
The container must be a community or acollection collection h4. h4. h4. h4. h2. h1. org.dspace.browse.BrowseException: The container must be a community or acollection at collection at org.dspace.browse.BrowseInfo.setBrowseContainer(BrowseInfo.java:167) at at org.dspace.browse.BrowseEngine.browseByItem(BrowseEngine.java:242) at at org.dspace.browse.BrowseEngine.browse(BrowseEngine.java:448) |
- Aside from the fact that the BrowseInfo reports browsing NOT in a community or collection as being in an Invalid Container, the very first primitive tests suggest that the application of the constraint code has not immediately broken anything else. Always a good start!
- The very first tentative test of the constraint code indicates that while the code is working, it appears to have missed the 5 items submitted later on. Since they should be in the same collection, there must be some sort of problem ... investigating.
- Actually, there appears to be a problem with the constraining process, so that constraining to collection does not take effect
- The reason for this is that the bit of code which tells which container you are in is clever enough to lift out the community /and/ the collection if you are in a collection. I'm not sure what happens when you are in a stack of communities. Perhaps I need to modify the code slightly to eliminate this danger.
- This all appears to happen somewhere high up the stack, possibly in the DSpaceServlet (I've decided not to track it any further). Anyway, the logic just needs to figure out that if we are in a collection then it doesn't need to bother with the community.
- The reason for this is that the bit of code which tells which container you are in is clever enough to lift out the community /and/ the collection if you are in a collection. I'm not sure what happens when you are in a stack of communities. Perhaps I need to modify the code slightly to eliminate this danger.
- With the above logic implemented, the collection browse functions apparently correctly (tests still to be fully carried out). Unfortunately, it still doesn't appear to work correctly for the community
- The table community_index_3 is a view on index_3 with additional community information. This table really doesn't contain the values that we want to see in the browse, so the problem is most likely in the indexing process itself (or, more likely still, in the community2item table)
- The problem is that there is one table called "communities2item" which contains only 22 of the 27 current records, and another called "community2item" which contains all the 27 records, and is a view on the data. The question is, why are there two so similarly named tables, and which one should we really be using, and why do they contain (slightly )different data.
- The "communities2item" table was used by the previous browse code, and was not being updated by the new indexer. I have decided to stick with the existing view "community2item" instead. With that change to the community_index_<n> views, everything appears to be selected. Next, on to test the new functionality ...
- Actually, there appears to be a problem with the constraining process, so that constraining to collection does not take effect
- It looks as though the Indexer might not be dropping old tables and views. Not quite sure why yet - one problem at a time.
...
- Create more top level communities
- Create some second level communities
- Create more collections as the second and third levels
- Add more items to the system ensuring there are results for all collections
- map items into more than one collection.
| Panelcode |
|---|
handle/123456789/2/browse?type=dateissued&order=ASC&rpp=10&sort_by=0 |
BrowseInfo String Representation: Browsing 1 to 11 of 40 in index: dateissued (data type: date, display type: full) ||
| Panelcode |
|---|
Browsing in collection: 1 (123456789/2)|| Listing over 3 columns: dc.date.issued,dc.title.null,dc.contributor.* on value: null|| Sorting by: dc.date.issued ASC(option 0)|| Item {{Item ID: 1 :: [dc.date.issued:2006-11-16T17.08.11Z][dc.title.null:Submit1Jones, Richard Item ID: 2 : 1][dc.contributor.*:Jones, Richard]}} {{Item ID: 2 :: [dc.date.issued:2006-11-16T17.08.42Z][dc.title.null:submit 2][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 3 :: [dc.date.issued:2006-11-16T17.09.05Z][dc.title.null:submit 3][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 4 :: [dc.date.issued:2006-11-16T17.09.26Z][dc.title.null:submit 4][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 5 :: [dc.date.issued:2006-11-16T17.09.52Z][dc.title.null:submit 5][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 6 :: [dc.date.issued:2006-11-16T17.10.18Zsubmit 6Jones, Richard Item ID: 7 :: 16T17.10.18Z][dc.title.null:submit 6][dc.contributor.*:Jones, Richard]}} {{Item ID: 7 :: [dc.date.issued:2006-11-16T17.10.43Z][dc.title.null:submit 7][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 8 :: [dc.date.issued:2006-11-16T17.11.08Z][dc.title.null:submit 8][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 9 :: [dc.date.issued:2006-11-16T17.11.30Z][dc.title.null:submit 9][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 10 :: [dc.date.issued:2006-11-16T17.11.56Z][dc.title.null:submit 10Jones, Richard Item ID: 11 : ][dc.contributor.*:Jones, Richard]}} {{Item ID: 11 :: [dc.date.issued:2006-11-16T17.12.19Z][dc.title.null:submit 11][dc.contributor.*:Jones, Richard]}}|| |
Failure Analysis: Although the result set appears to be correct, there are not 40 elements in this collection (which is the correct collection). This is evidently a missing statement in the WHERE clause of the count mechanism.
| Panelcode |
|---|
BrowseInfo String Representation: Browsing 1 to 11 of 27 in index: dateissued (data type: date, display type: full) || Browsing in collection: 1 (123456789/2)|| Listing over 3 columns: dc.date.issued,dc.title.null,dc.contributor.* on value: null|| Sorting by: dc.date.issued ASC(option 0)|| Item {{Item ID: 1 :: [dc.date.issued:2006-11-16T17.08.11Z][dc.title.null:Submit 1][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 2 :: [dc.date.issued:2006-11-16T17.08.42Z][dc.title.null:submit 2][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 3 :: [dc.date.issued:2006-11-16T17.09.05Z][dc.title.null:submit 3][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 4 :: [dc.date.issued:2006-11-16T17.09.26Z][dc.title.null:submit 4][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 5 :: [dc.date.issued:2006-11-16T17.09.52Z][dc.title.null:submit 5][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 6 :: [dc.date.issued:2006-11-16T17.10.18Z][dc.title.null:submit 6][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 7 :: [dc.date.issued:2006-11-16T17.10.43Zsubmit 7Jones, Richard Item ID: 8 :: .10.43Z][dc.title.null:submit 7][dc.contributor.*:Jones, Richard]}} {{Item ID: 8 :: [dc.date.issued:2006-11-16T17.11.08Z][dc.title.null:submit 8][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 9 :: [dc.date.issued:2006-11-16T17.11.30Z][dc.title.null:submit 9][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 10 :: [dc.date.issued:2006-11-16T17.11.56Z][dc.title.null:submit 10][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 11 :: [dc.date.issued:2006-11-16T17.12.19Z][dc.title.null:submit 11][dc.contributor.*:Jones, Richard]}}|| |
Success Metric: right number of results, range, total, right collection, ordered by date ascending
| Panelcode |
|---|
handle/123456789/32/browse?type=title&order=DESC&rpp=5&sort_by=0 |
BrowseInfo String Representation: Browsing 1 to 3 of 3 in index: title (data type: title, display type: full) ||
| Panelcode |
|---|
Browsing in collection: 2 (123456789/32)|| Listing over 3 columns: dc.date.issued,dc.title.null,dc.contributor.* on value: null|| Sorting by: dc.title DESC(option 0)|| Item {{Item ID: 29 :: [dc.date.issued:2006-12-04T11.55.38Z][dc.title.null:Submit G][dc.contributor.*:Garrison,Gertrude Item Gertrude]}} {{Item ID: 28 :: [dc.date.issued:2006-12-04T11.55.08Z][dc.title.null:Submit F][dc.contributor.*:Frankfurt,Freddie Item Freddie]}} {{Item ID: 1 :: [dc.date.issued:2006-11-16T17.08.11Z][dc.title.null:Submit 1][dc.contributor.*:Jones, Richard]}}|| |
Success Metric: correct range and result count, sorted by title descending, limited to correct collection
| Panelcode |
|---|
handle/123456789/34/browse?type=author&order=ASC&rpp=5&sort_by=0 |
BrowseInfo String Representation: Browsing 1 to 3 of 3 in index: author (data type: text, display type: single) ||
| Panelcode |
|---|
Browsing in collection: 3 (123456789/34)|| Listing single column: dc.contributor.* on value: null|| Sorting by: dc.contributor.* ASC(option 0)|| { \{ \{Value: Harrison, Harry\}} { \} \{ \{Value: Ianson, Irene\}} { \} \{ \{Value: Jones, Richard\}\}|| |
Success Metric: correct range and result count and results, sorted by contributor ascending, limited to correct collection
| Panelcode |
|---|
handle/123456789/38/browse?type=subject&order=DESC&rpp=5&sort_by=0 |
Success Metric: no results, as expected (no items in this collection have subjects
| Panelcode |
|---|
handle/123456789/1/browse?type=dateissued&order=DESC&rpp=5&sort_by=1&focus=20 |
BrowseInfo String Representation: Browsing 15 to 20 of 27 in index: dateissued (data type: date, display type: full) ||
| Panelcode |
|---|
Browsing in community: 1 (123456789/1)|| Listing over 3 columns: dc.date.issued,dc.title.null,dc.contributor.* on value: submit 20|| Sorting by: dc.title DESC(option 1)|| Item {{Item ID: 20 :: [dc.date.issued:2006-11-16T17.15.42Z][dc.title.null:submit 20][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 2 :: [dc.date.issued:2006-11-16T17.08.42Zsubmit 2Jones, Richard Item ID: 19 :: 16T17.08.42Z][dc.title.null:submit 2][dc.contributor.*:Jones, Richard]}} {{Item ID: 19 :: [dc.date.issued:2006-11-16T17.15.19Z][dc.title.null:submit 19][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 18 :: [dc.date.issued:2006-11-16T17.14.58Z][dc.title.null:submit 18][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 17 :: [dc.date.issued:2006-11-16T17.14.37Z][dc.title.null:submit 17][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 16 :: [dc.date.issued:2006-11-16T17.14.16Z][dc.title.null:submit 16][dc.contributor.*:Jones, Richard]}}|| |
Success Metric: correct range and result count, starting from item id 20, sorted by title descending, limited to correct community
NOTE: during testing it became clear that the browse URL not only needs to be validated for structure, but also that sort_by (for one example) is sensitive to the value passed, therefore we need to build in value validation also
| Code Block |
|---|
|
| Panel |
handle/123456789/33/browse?type=title&order=ASC&rpp=5&sort_by=2&focus=30 |
BrowseInfo String Representation: Browsing 3 to 6 of 6 in index: title (data type: title, display type: full) ||
| Panelcode |
|---|
Browsing in community: 3 (123456789/33)|| Listing over 3 columns: dc.date.issued,dc.title.null,dc.contributor.* on value: 2006-12-04t11:56:30z|| Sorting by: dc.date.issued ASC(option 2)|| Item {{Item ID: 30 :: [dc.date.issued:2006-12-04T11.56.30Z][dc.title.null:Submit H][dc.contributor.*:Harrison,Harry Item Harry]}} {{Item ID: 31 :: [dc.date.issued:2006-12-04T11.57.01Z][dc.title.null:Submit I][dc.contributor.*:Ianson,Irene Item Irene]}} {{Item ID: 32 :: [dc.date.issued:2006-12-04T11.57.30Z][dc.title.null:Submit J][dc.contributor.*:Johnson,James Item James]}} {{Item ID: 33 :: [dc.date.issued:2006-12-04T11.57.50Z][dc.title.null:Submit K][dc.contributor.*:Karlson, Karl]}}|| |
Success Metric: correct range and result count, starting from item id 30, sorted by date issued ascending, limited to correct community
| Panelcode |
|---|
handle/123456789/33/browse?type=author&order=DESC&value=Jones%2C+Richard&vfocus=submit+2&rpp=5&sort_by=2 |
BrowseInfo String Representation: Browsing 1 to 2 of 2 in index: author (data type: text, display type: single) ||
| Panelcode |
|---|
Browsing in community: 3 (123456789/33)|| Listing single column: dc.contributor.* on value: submit 2|| Sorting by: dc.date.issued DESC(option 2)|| Item {{Item ID: 9 :: [dc.date.issued:2006-11-16T17.11.30Z][dc.title.null:submit 9][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 4 :: [dc.date.issued:2006-11-16T17.09.26Z][dc.title.null:submit 4][dc.contributor.*:Jones, Richard]}}|| |
Success Metric: correct range and result count, displaying only author "Jones, Richard", focussing on value submit 2 (this doesn't exist in the list, so the result set is correct), sorted by date issued descending, limited to correct community. It is interesting to note that these are both "mapped" items.
| Panelcode |
|---|
handle/123456789/33/browse?type=subject&order=ASC&rpp=5&value=asdf&sort_by=0 |
BrowseInfo String Representation: Browsing 1 to 2 of 2 in index: subject (data type: text, display type: single) ||
| Panelcode |
|---|
Browsing in community: 3 (123456789/33)|| Listing single | Listing single column: dc.subject.* on value: null|| Sorting by: dc.subject.* ASC(option 0)|| Item {{Item ID: 4 :: [dc.date.issued:2006-11-16T17.09.26Z][dc.title.null:submit 4][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 9 :: [dc.date.issued:2006-11-16T17.11.30Z][dc.title.null:submit 9][dc.contributor.*:Jones, Richard]}}|| |
Success Metric: correct range and result set, displaying only items (mapped in) with the subject asdf (although for some reason the BrowseInfo is not reporting on the value), sorted by subject ascending (doesn't really make much sense)
Testing Browse URL Summary (part 3)
border="1"
| Panelcode |
|---|
A summary of the URL parameters tested above- ! test |- ! test !! mode !! type !! order !! value !! focus !! vfocus !! rpp !! sort_by !! starts_with !! collection !! community |
- | 18 | dateissued | ASC | N/A | 10 | 0 | /2 | N/A |
- | 19 | Full/Item | title | DESC | N/A | 5 | 0 | /32 | N/A |
- | 20 | Single/Value | author | ASC | 5 | 0 | /34 | N/A |
- | 21 | Single/Value | subject | DESC | 5 | 0 | /38 | N/A |
- | 22 | Full/Item | dateissued | DESC | N/A | 20 | 5 | 1 | N/A | /1 |
- | 23 | Full/Item | title | ASC | N/A | 30 | 5 | 2 | N/A | /33 |
- | 24 | Value | author | DESC | Jones, Richard | submit 2 | 5 | 2 | N/A | /37 |
- | 25 | Value | subject | ASC | asdf | 5 | 0 | N/A | /31
Progress Update: 04-12-2006, 13:15 GMT
...
A flaw has been found in the code that determines the current position of the first item to be displayed. This arises because the code which generates the query does not take into account the value being browsed on, and does not correctly negate the SQL query to obtain the relevant position. For example, the following SQL generates a valid result set:
| Panelcode |
|---|
SELECT * FROM index_2 WHERE sort_1 <= 'submit 13' AND sort_value = 'jones, richard' ORDER BY sort_1 DESC LIMIT 6 |
The query to obtain the position of the start pointer generated is as follows:
| Panelcode |
|---|
SELECT COUNT(DISTINCT(value)) AS number FROM index_2 WHERE sort_1 > 'submit 13' |
This is incorrect. Instead the query ought to read:
| Panelcode |
|---|
SELECT COUNT (*) FROM index_2 WHERE sort_1 > 'submit 13' AND sort_value = 'jones, richard'; |
Note here how although the direction of the comparator is correct, no sort_value was specified in the original query, and it also does a SELECT DISTINCT with insufficient cause. Some modification to the browse engine will fix this reasonably quickly.
...
- Although we have run all the browse tests again, we have not posted here the results. For example, this is the sort of thing that we are now seeing reported by the BrowseInfo object:
| Panelcode |
|---|
browse?type=author&order=ASC&rpp=10&sort_by=1&starts_with=submit+2&value=Jones%2C+Richard |
BrowseInfo String Representation: Browsing 12 to 21 of 22 in index: author (data type: text, display type: single) ||
| Panelcode |
|---|
Browsing in all of DSpace: no id available/necessary|| Listing single column: dc.contributor.* sort column starting with: submit 2|| Sorting by: dc.title ASC (option 1)|| Item {{Item ID: 2 :: [dc.date.issued:2006-11-16T17.08.42Z][dc.title.null:submit 2][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 20 :: [dc.date.issued:2006-11-16T17.15.42Z][dc.title.null:submit 20][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 21 :: [dc.date.issued:2006-11-16T17.16.02Z][dc.title.null:submit 21][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 22 :: [dc.date.issued:2006-11-16T17.17.24Z][dc.title.null:submit 22][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 3 :: [dc.date.issued:2006-11-16T17.09.05Z][dc.title.null:submit 3][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 4 :: [dc.date.issued:2006-11-16T17.09.26Z][dc.title.null:submit 4][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 5 :: [dc.date.issued:2006-11-16T17.09.52Z][dc.title.null:submit 5][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 6 :: [dc.date.issued:2006-11-16T17.10.18Zsubmit 6Jones, Richard Item ID: 7 : 16T17.10.18Z][dc.title.null:submit 6][dc.contributor.*:Jones, Richard]}} {{Item ID: 7 :: [dc.date.issued:2006-11-16T17.10.43Z][dc.title.null:submit 7][dc.contributor.*:Jones,Richard Item Richard]}} {{Item ID: 8 :: [dc.date.issued:2006-11-16T17.11.08Z][dc.title.null:submit 8][dc.contributor.*:Jones, Richard]}}|| Top of next page: Item ID: 9;Top of previous page: Item ID: 10|| |
As you can see, it now presents the next and previous page values, and the ones reported above are correct.
...
OK, so now we have enough confidence to invest some time building the UI for this browse code. We have already attacked the logic of the display to generate the strings used for debug above, so it will be a matter of applying that logic to a nicer interface, with a variety of widgets. There is no need to significantly redesign the Browse UI, so the following outlines are almost identical except in some minor ways, to the existing interface.Single/Value Browseh1. When we are looking at a single value browse, such as "by author", the page would look as follows:
!browse-ui-single.png! Full/Item & Value Browseh1. When looking at a list of actual items, such as "by title" or we are looking at a list of items associated with a value browse such as "items by author X", the page would look as follows:
! browse-ui-full.png 
!Navigation Elementsh1. These two navigation elements will be used on both the above pages. The first will be used whenever the type is "title" or "text" and the second when the type is "date". This will also be dependent on which value we are sorting by (i.e. whether it's text or date) 
Still More Programmer's Notes
...
- The next and previous links are going to require only a subset of all the actual parameters that can be passed into the browse. They should both be of the form:
| Code Block |
|---|
[ |
| Panel |
handle-_prefix_-_suffix_-]browse?type=<type>&sort_by=<sort_by>&order=<order>[&.value=_value_][&.rpp=_rpp_][&[focus=<focus>|vfocus=_vfocus_]] |
- Woohoo! We can now see the next and previous links operating against a live browse listing. Time to call it a day.
...
BUG: starts_with for 2006-11 order ASC gives everything after 2006-11, NOT inclusive of it. This isn't really correct (certainly not expected behaviour. It ought to start with the beginning of Nov 2006, not Dec 2006 as it does. This was using the query:
| Panelcode |
|---|
browse?type=dateissued&order=ASC&rpp=10&sort_by=0&starts_with=2006-11 |
- This problem is actually related to the way that year and month become starts_with. In order to solve this problem we must subtract 1 from the month number when order is ASC and leave it as-is when order is DESC
...
- This is the tip of an iceberg. The bug is due to the fact that the following expression does not evaluate to true:
| Panelcode |
|---|
if (sortOrder.isDate() || (browseIndex.isDate() && sortOrder.isDefault())) |
We have not, according to this, selected a sortOrder that works by date, even though we are jumping to a date browse. The broken bit is in two bits, depending on how you look at it.
- The trivial way this is broken is that the UI should enforce a sort_by parameter on the value browse. If sort_by = 2 is appended to the URL for a value browse, then it evaluates correctly. The error arises because there is no sort_by specified. See the next point
- Deep down, the following SQL is executed:
| Panelcode |
|---|
SELECT * FROM index_2 WHERE sort_value >= '2006-10' AND sort_value = 'jones, richard' ORDER BY sort_value ASC LIMIT 21 |
As you can see, it is asking two mutually exclusive (in terms of our semantics) things of the sort_value column. This is the symptom of there being no sort_by order specified, and the engine defaulting to the sort_value column
...
All Browse Indexes will be looked up as follows:
| Panelcode |
|---|
browse.type.<type> = <value> |
for example
| Panelcode |
|---|
browse.type.author = Author |
All page content for the Full/Item and Item/Value browse will be looked up as follows:
| Panelcode |
|---|
browse.full.<content> = <value> |
for example
| Panelcode |
|---|
browse.full.range = Showing Results \{0\} to \{1\} of \{2\} |
All page content for Single/Value browse will be looked up as follows:
| Panelcode |
|---|
browse.single.<content> = <value> |
for example
| Panelcode |
|---|
browse.single.order = Order |
Further to this, the page title will have the i18n key
| Panelcode |
|---|
browse.page-title |
And the elements for the standard navigations at the top of each browse page will have they form
| Panelcode |
|---|
browse.nav.<element> = <value> |
for example
| Panelcode |
|---|
browse.nav.jump-to = Jump to a point in the index: |
Elements for the no results page will have the form
| Panelcode |
|---|
browse.no-results.<content> = <value> |
Elements that will appear in the left navigation will have the form
| Panelcode |
|---|
browse.menu.<type> = <value> |
Progress Update: 11-12-2006, 16:00 GMT
...
Download the attached file to see the output of the first 27,000 items indexed in my test environment. This operation ran as an indexing job for over 15 hours. The data file consists of 4 columns: the integer number of the index job, the unix time in milliseconds of the start of the operation, the item id, and the time it took to index. Plot this with gnuplot to see what's going on:
| Panelcode |
|---|
plot "browse.uat.dat" u 1:4 w linespoints |
!Browse.uat.dat!</html>