C# – split huge 40000 page pdf into single pages, itextsharp, outofmemoryexception

asp.netcitextsharpout-of-memorypdf

I am getting huge PDF files with lots of data. The current PDF is 350 MB and has about 40000 pages. It would of course have been nice to get smaller PDFs, but this is what I have to work with now 🙁

I can open it in acrobat reader with some delay when loading but after that acrobat reader is quick.

Now I need to split the huge file into single pages, then try to read some recipient data from the pdf pages, and then send the one or two pages that each recipient should get to each particular recipient.

Here is my very small code so far using itextsharp:

var inFileName = @"huge350MB40000pages.pdf";
PdfReader reader = new PdfReader(inFileName);
var nbrPages = reader.NumberOfPages;
reader.Close();

What happens is it comes to the second line "new PdfReader" then stays there for perhaps 10 minutes, the process gets to about 1.7 GB in size, and then I get an OutOfMemoryException.

I think the "new PdfReader" attempts to read the entire PDF into memory.

Is there some other/better way to do this?
For example, can I somehow read only a part of a PDF file into memory instead of all of it at once?
Could it work better using some other library than itextsharp?

Best Answer

From what I have read, it looks like when instantiating the PdfReader that you should use the constructor that takes in a RandomAccessFileOrArray object. Disclaimer: I have not tried this out myself.

iTextSharp.text.pdf.PdfReader reader = new iTextSharp.text.pdf.PdfReader(new iTextSharp.text.pdf.RandomAccessFileOrArray(@"C:\PDFFile.pdf"), null);

Related Solutions

Ios – Fast and Lean PDF Viewer for iPhone / iPad / iOS – tips and hints

I have build such kind of application using approximatively the same approach except :

I cache the generated image on the disk and always generate two to three images in advance in a separate thread.
I don't overlay with a UIImage but instead draw the image in the layer when zooming is 1. Those tiles will be released automatically when memory warnings are issued.

Whenever the user start zooming, I acquire the CGPDFPage and render it using the appropriate CTM. The code in - (void)drawLayer: (CALayer*)layer inContext: (CGContextRef) context is like :

CGAffineTransform currentCTM = CGContextGetCTM(context);    
if (currentCTM.a == 1.0 && baseImage) {
    //Calculate ideal scale
    CGFloat scaleForWidth = baseImage.size.width/self.bounds.size.width;
    CGFloat scaleForHeight = baseImage.size.height/self.bounds.size.height; 
    CGFloat imageScaleFactor = MAX(scaleForWidth, scaleForHeight);

    CGSize imageSize = CGSizeMake(baseImage.size.width/imageScaleFactor, baseImage.size.height/imageScaleFactor);
    CGRect imageRect = CGRectMake((self.bounds.size.width-imageSize.width)/2, (self.bounds.size.height-imageSize.height)/2, imageSize.width, imageSize.height);
    CGContextDrawImage(context, imageRect, [baseImage CGImage]);
} else {
    @synchronized(issue) { 
        CGPDFPageRef pdfPage = CGPDFDocumentGetPage(issue.pdfDoc, pageIndex+1);
        pdfToPageTransform = CGPDFPageGetDrawingTransform(pdfPage, kCGPDFMediaBox, layer.bounds, 0, true);
        CGContextConcatCTM(context, pdfToPageTransform);    
        CGContextDrawPDFPage(context, pdfPage);
    }
}

issue is the object containg the CGPDFDocumentRef. I synchronize the part where I access the pdfDoc property because I release it and recreate it when receiving memoryWarnings. It seems that the CGPDFDocumentRef object do some internal caching that I did not find how to get rid of.

ITextSharp-generated PDFs now cause Save dialog in Adobe Reader X

The problem is this line:

Response.OutputStream.Write(MS.GetBuffer(), 0, MS.GetBuffer().Length)

The GetBuffer method returns the entire internal buffer which is larger that the actual content. The bad PDF has about 10kb of garbage content at the end (bytes of zero), the good PDF has only a few garbage bytes. Use the ToArray() method of the memory stream to get the PDF file and the problem will be fixed. You will also get smaller files.

byte[] pdf = MS.ToArray();
Response.OutputStream.Write(pdf, 0, pdf.Length);

Also set the "Content-Length" with the length of the pdf array.

Related Topic