Uploaded image for project: 'Bitrepository'
  1. Bitrepository
  2. BITMAG-619

The system should scale to large number of files

    XMLWordPrintable

Details

    • Improvement
    • Resolution: Rejected
    • Critical
    • Continuous
    • None
    • Test
    • None

    Description

      The current system will properly hit a number of bottlenecks as the number of files increases. Critical functionality could be:

      Pr. file operations on lists of files

      If methods are implemented as:

      for (Object file: Filelist) {
         performOperationOnReferenceClient();  
      }
      

      This will not scale, and may potential kill a collection. Therefore code like this must be refactored to use bulk operations.

      Avoid fileList iteration in general

      This is a generelization of the rule described above. Constructs methods like:

      for (Object file: Filelist) {
         method();  
      }
      

      should be avoided in general, as the filelists can grow to be very large. Try to limit the list by SQL, or move the functionality entirely to SQL.

      Lists in webclient

      All lists displayed should be limited. If the lists can grow to arbitary sizes, this will a some point make the webclient unusable. The solution to this could be multi step.

      1. All lists in the web client have hardcoded limits on the size of lists, so only the first NNN elements are show, potentially with a message informing the user that the the number of elements exceeds the list limit.
        Filtering functionality must therefore be used to limit lists, if all elements needs to be viewed.
      2. The WS requests should be extended with parameter for indicating the maximum number of elements which should be returned.
      3. Implement paged list. That is it is possible to page through subsets of the full list for a given query, eg. like googles search functionality.

      Attachments

        Activity

          People

            mss Mikis Seth Sørensen (Inactive)
            mss Mikis Seth Sørensen (Inactive)
            Mikis Seth Sørensen Mikis Seth Sørensen (Inactive)
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: