|
Speed up performance when you do mutiple document.removeFromFolder() or document.putInFolder()
Julian Buss, May 10th, 2010 16:23:54
Tags: Development
this should be a no-brainer for all LotusScript experts, but I just came across it and thought I can share it: I have an agent which runs through all documents of a view, and removes each document from a folder and puts it in another folder, depending on some criterias.
The loop basically looked like this: --- while (not doc is nothing) ... if (...) then call doc.removeFromFolder("somefolder") call doc.putInFolder("anotherfolder") end if ... wend --- works fine, until you have more than a couple of hundred documents. In my case, I was faced with over 100.000 documents, which caused the agent to take a loooooooong time to run (or to be exact: it was terminated by the amgr after 15 minutes). Solution for this problem is to collect all documents that should be removed from a folder or put into a folder in a NotesDocumentCollection, and remove/put them all at once using NotesDocumentCollection.removeAllFromFolder() / NotesDocumentCollection.putAllInFolder(). The new loop likes like this: --- Dim dc as notesDocumentCollection set dc = session.currentDatabase.createDocumentCollection() 'works for Notes 8 and higher while (not doc is nothing) ... if (...) then call dc.addDocument(doc) end if ... wend call dc.removeAllFromFolder("somefolder") call dc.putAllInFolder("anotherfolder)" --- Pretty simple and runs so much faster. P.S.: if you are working with really many document you may want to "flush" the DocumentCollection every 1000 documents or so, so that in case the agent breaks down you don't loose all the work.
Comments (5) | Permanent Link
1) Speed up performance when you do mutiple document.removeFromFolder() or document.putInFolder()
"...so that in case the agent breaks down you don't loose all the work. " Julian this is a great tip but sadly most of the reasons I have for moving things in and out of folders is to signify the successful processing of something. If the code breaks while looping through the documents, and a large number of "processed" documents are still in the original folder, you'll end up re-processing them when you re-run the agent. Now, if the processing of the documents involves *changing* the documents themselves, then you could have a separate process cleanup any stragglers, but then you're talking more complexity. 2) Speed up performance when you do mutiple document.removeFromFolder() or document.putInFolder()
Kevin, sure, it depends on the usecase. In my case I moved all docs of the DocumentCollection out of the folder and into the other folder every 500th time. You can change that to every 100th oder 50th or so, and I guess you still would have performance advantages. 3) How to flush periodically?
Guten tag, Julius! Just to round out your example, I'd suggest adding a few lines of code indicating how you go about flushing every 100th or 50th document. (Maybe with a Print statment thrown in that logs each flush on the status line?) ... Cheers, Tony. 4) Reason for adding the Print statement
I should have stated the obvious, but adding that Print statement could facilitate recovery if the agent crashes part way through. 5) Speed up performance when you do mutiple document.removeFromFolder() or document.putInFolder()
here is some more code: --- Dim dc as notesDocumentCollection dim i as long set dc = session.currentDatabase.createDocumentCollection() 'works for Notes 8 and higher i = 0 while (not doc is nothing) ... if (...) then call dc.addDocument(doc) i = i + 1 if i mod 500 = 0 then ' flush collection call dc.removeAllFromFolder("somefolder") call dc.putAllInFolder("anotherfolder)" delete dc set dc = session.currentDatabase.createDocumentCollection() end if end if ... wend call dc.removeAllFromFolder("somefolder") call dc.putAllInFolder("anotherfolder)" --- |
You can hire me.
See my Linkedin profile for details.
Get in touch: Thanks for reading and have a nice time here! Please note my Apps for iPhone and iPad:
NotesBook: takes your Lotus Notes Notebook (Journal) to your iPhone and iPad
xpageswiki.com: huge XPages Tips & Tricks collection for iPhone and iPad
|
