From PHP to Go: How We Made Excel Streaming 5x Faster and Cut Memory Usage by 95%

The Unexpected Journey of Rewriting a Production-Ready Package#

After successfully building a PHP package that streams millions of rows to Excel, I thought the problem was solved. Then I tried it in Go and everything changed.

The Story So Far#

A few months ago, I shared how we built kolay-xlsx-stream — a PHP package that could export 4.5 million rows to Excel using just 178MB of memory. The response was overwhelming: hundreds of developers were facing the same problem.

But then someone asked: “Have you tried this in Go?”

Challenge accepted. 🚀

Why Rewrite in Go?#

I’ll be honest — I love PHP. Laravel is my go-to for rapid development. But when it comes to:

• CPU-intensive operations (ZIP compression, XML generation)
• Concurrent processing (handling multiple exports simultaneously)
• Memory efficiency (garbage collection, data structures)
• Deployment simplicity (single binary vs PHP runtime)

Go has some serious advantages.

The Numbers Don’t Lie#

Here’s what happened when we rewrote the package in Go:

PHP vs Go: Head-to-Head Benchmark#

For 2 million rows:

• Speed: 12x faster (41s → 3.5s)
• Memory: 95% less (178MB → 2MB)
• Throughput: 600,000 rows/second vs 50,000 rows/second

The Architecture: Built for Performance#

1. The Sink Pattern#

Just like the PHP version, we use a Sink abstraction:

type Sink interface {
    io.Writer
    io.Closer
}
// File sink
sink, _ := kolayxlsxstream.NewFileSink("output.xlsx")
// S3 sink - streams directly to AWS
sink, _ := kolayxlsxstream.NewS3Sink(ctx, s3Client, bucket, key)

The beauty? The writer doesn’t care where data goes. File, S3, memory buffer — it’s all the same.

2. Zero Memory Allocations in Hot Path#

Go’s compiler is smart. Really smart. Here’s how we write rows with zero allocations:

func (w *Writer) WriteRow(row []interface{}) error {
    // Pre-allocated buffer
    w.rowBuffer.Reset()
    // Write XML directly to buffer (no string concatenation)
    w.rowBuffer.WriteString("<row r=\"")
    w.rowBuffer.WriteString(strconv.Itoa(w.currentRow))
    w.rowBuffer.WriteString("\">")
    // Cells written directly
    for colIdx, value := range row {
        w.writeCellValue(colIdx, value)
    }
    w.rowBuffer.WriteString("</row>")
    // Single write to sink
    _, err := w.sink.Write(w.rowBuffer.Bytes())
    return err
}

Result: Processing 1 million rows creates zero garbage collection pressure.

3. Concurrent Compression#

Go’s compress/flate package is crazy fast because it can use all CPU cores:

config := kolayxlsxstream.DefaultConfig()
config.CompressionLevel = 1  // Fast compression

writer := kolayxlsxstream.NewWriter(sink, config)

On an M4 chip, this uses all performance cores automatically.

Direct S3 Streaming: The Real Game Changer#

The S3 implementation deserves its own section.

The Challenge#

AWS S3 multipart upload requires:

• Minimum 5MB per part (except last part)
• Managing upload IDs and ETags
• Handling failures gracefully
• Assembling parts correctly

The Solution#

type S3Sink struct {
    client         S3ClientAPI
    bucket         string
    key            string
    ctx            context.Context
    uploadID       *string
        buffer         *bytes.Buffer
        partNumber     int32
        completedParts []types.CompletedPart
    }
    func (s *S3Sink) Write(p []byte) (n int, err error) {
        n, err = s.buffer.Write(p)
        s.totalBytes += int64(n)
        // Auto-upload when buffer hits 32MB
        if s.buffer.Len() >= int(s.options.PartSize) {
            if err := s.uploadPart(); err != nil {
                return n, fmt.Errorf("failed to upload part: %w", err)
            }
        }
        return n, err
    }

Memory Pattern During S3 Upload#

Unlike PHP where memory grows linearly, Go maintains constant memory:

Memory (MB)
    ↑
35  │  ████████████████████████████████
    │  ████████████████████████████████
    │  ████████████████████████████████
 2  │  ████████████████████████████████
    └─────────────────────────────────→ Time
         (Constant ~33MB average)

Why 33MB? One 32MB buffer for S3 multipart + ~1MB for ZIP structures.

Real-World Usage: Laravel Integration#

Yes, you can use this from Laravel! Here’s how:

// Laravel controller
use Symfony\Component\Process\Process;
public function exportToS3()
{
    // Call Go binary
    $process = new Process([
        '/usr/local/bin/xlsx-exporter',
        '--bucket', 'my-bucket',
        '--key', 'exports/users.xlsx',
        '--query', 'SELECT * FROM users'
    ]);
    $process->setTimeout(300);
    $process->start();
    return response()->json([
        'status' => 'processing',
        'pid' => $process->getPid()
    ]);
}

Or Use Go as a Microservice#

// Go service
http.HandleFunc("/export", func(w http.ResponseWriter, r *http.Request) {
    bucket := r.FormValue("bucket")
    key := r.FormValue("key")
    // Create S3 sink
    sink, _ := kolayxlsxstream.NewS3Sink(ctx, s3Client, bucket, key)
    writer := kolayxlsxstream.NewWriter(sink)
    // Stream from database
    rows, _ := db.Query("SELECT * FROM large_table")
    defer rows.Close()
    writer.StartFile([]interface{}{"ID", "Name", "Email"})
    for rows.Next() {
        var id int
        var name, email string
        rows.Scan(&id, &name, &email)
        writer.WriteRow([]interface{}{id, name, email})
    }
    stats, _ := writer.FinishFile()
    json.NewEncoder(w).Write(stats)
})

Testing: 82% Coverage with S3 Mocks#

One of the challenges with S3 testing is… well, S3. We solved this with interfaces:

type S3ClientAPI interface {
    CreateMultipartUpload(ctx context.Context, params *s3.CreateMultipartUploadInput, optFns ...func(*s3.Options)) (*s3.CreateMultipartUploadOutput, error)
    UploadPart(ctx context.Context, params *s3.UploadPartInput, optFns ...func(*s3.Options)) (*s3.UploadPartOutput, error)
    CompleteMultipartUpload(ctx context.Context, params *s3.CompleteMultipartUploadInput, optFns ...func(*s3.Options)) (*s3.CompleteMultipartUploadOutput, error)
    AbortMultipartUpload(ctx context.Context, params *s3.AbortMultipartUploadInput, optFns ...func(*s3.Options)) (*s3.AbortMultipartUploadOutput, error)
}

Now we can mock S3 completely:

func TestS3Upload(t *testing.T) {
    mockClient := &MockS3Client{
        uploadFunc: func(ctx context.Context, params *s3.UploadPartInput) (*s3.UploadPartOutput, error) {
            return &s3.UploadPartOutput{
                ETag: aws.String("mock-etag"),
            }, nil
        },
    }
    sink, _ := kolayxlsxstream.NewS3Sink(ctx, mockClient, "test-bucket", "test-key")
    // ... test without real S3!
}

Result: 19 tests, all passing, no AWS credentials needed.

Deployment: Single Binary vs PHP Runtime#

PHP Deployment#

# Server needs:
- PHP 8.1+
- Extensions: zip, xml, mbstring
- Composer dependencies
- 512MB+ memory_limit

Go Deployment#

# Build once
GOOS=linux GOARCH=amd64 go build -o xlsx-exporter
# Deploy anywhere - just copy the binary
scp xlsx-exporter user@server:/usr/local/bin/
# Run
./xlsx-exporter --help

That’s it. No runtime, no dependencies, no configuration.

Benchmark Deep Dive: S3 Upload#

We tested uploading files directly to AWS S3:

For 2M rows to S3:

• PHP: 112 seconds (17,800 rows/sec)
• Go: 22 seconds (89,400 rows/sec)
• 5x faster upload

Memory Efficiency: The O(1) Promise#

Both PHP and Go versions promise constant memory, but Go delivers it better:

PHP Memory Pattern (4.5M rows → S3)#

Memory peaks at 178MB average
Fluctuates ±50MB during multipart uploads
GC runs every ~30 seconds

Go Memory Pattern (2M rows → S3)#

Memory stays at ~2-5MB (!!!)
S3 buffer adds 32MB (one-time)
Total: ~33MB constant
GC barely runs

Why the difference?

• Go’s strings are immutable and cheap to copy
• PHP arrays have overhead (hash tables)
• Go’s garbage collector is generational and concurrent

Production Lessons Learned#

1. HeadBucket Permission Issues#

AWS IAM policies sometimes don’t include s3:HeadBucket. We handle this gracefully:

_, err = client.HeadBucket(ctx, &s3.HeadBucketInput{Bucket: bucket})
if err != nil {
    fmt.Println("⚠️  Cannot verify bucket (will try upload anyway)")
} else {
    fmt.Println("✅ Bucket accessible")
}

2. Compression Level Matters#

Recommendation: Use level 1 for speed, level 6 only if size matters.

3. Batch Writing is Key#

// Slow: Write one row at a time
for _, row := range rows {
    writer.WriteRow(row)
}
// Fast: Batch writes
batch := make([][]interface{}, 1000)
for i, row := range rows {
    batch[i%1000] = row
    if (i+1)%1000 == 0 {
        writer.WriteRows(batch)
    }
}

Result: 2x faster for large datasets.

Installation & Quick Start#

Install#

go get github.com/turgutahmet/kolayxlsxstream@v1.0.0

Basic Usage#

package main
import "github.com/turgutahmet/kolayxlsxstream"
func main() {
    // Create file sink
    sink, _ := kolayxlsxstream.NewFileSink("output.xlsx")
    writer := kolayxlsxstream.NewWriter(sink)
    // Write data
    writer.StartFile([]interface{}{"Name", "Email", "Score"})
    // Stream from anywhere
    for _, user := range users {
        writer.WriteRow([]interface{}{
            user.Name,
            user.Email,
            user.Score,
        })
    }
    stats, _ := writer.FinishFile()
    fmt.Printf("Exported %d rows in %.2fs\n", stats.TotalRows, stats.Duration)
}

S3 Streaming#

import (
    "github.com/aws/aws-sdk-go-v2/config"
    "github.com/aws/aws-sdk-go-v2/service/s3"
)
// Load AWS config
cfg, _ := config.LoadDefaultConfig(ctx)
client := s3.NewFromConfig(cfg)
// Create S3 sink
sink, _ := kolayxlsxstream.NewS3Sink(ctx, client, "my-bucket", "exports/data.xlsx")
// Same writer API!
writer := kolayxlsxstream.NewWriter(sink)
// ... write data

What’s Different from PHP?#

1. Type Safety#

// Go forces you to be explicit
writer.WriteRow([]interface{}{
    42,              // int
    "John Doe",      // string
    3.14159,         // float64
    true,            // bool
})
// PHP auto-converts (sometimes incorrectly)
$writer->writeRow([
    42,              // might become "42"
    "John Doe",
    3.14159,         // might lose precision
    true,            // might become 1
]);

2. Error Handling#

// Go: Explicit error checks
stats, err := writer.FinishFile()
if err != nil {
    log.Fatalf("Export failed: %v", err)
}
// PHP: Exceptions (can be missed)
try {
    $stats = $writer->finishFile();
} catch (\Exception $e) {
    // Easy to forget this
}

3. Concurrency#

// Go: Export multiple files concurrently
var wg sync.WaitGroup
for i := 0; i < 10; i++ {
    wg.Add(1)
    go func(idx int) {
        defer wg.Done()
        exportToExcel(idx)
    }(i)
}
wg.Wait()

This would crash PHP. In Go, it just works.

Community Response#

Since releasing v1.0.0:

• ⭐ Growing GitHub stars
• 📦 Indexed on pkg.go.dev
• 🏆 A+ on Go Report Card
• ✅ 82.4% test coverage
• 🚀 Zero issues reported

The Economics: PHP vs Go for Excel Exports#

Let’s talk money.

AWS Lambda Costs (1M row export, 100 times/day)#

PHP Lambda:

• Memory needed: 512 MB
• Execution time: 19.4s
• Cost per invocation: $0.0032
• Monthly cost: ~$960

Go Lambda:

• Memory needed: 128 MB
• Execution time: 1.67s
• Cost per invocation: $0.0002
• Monthly cost: ~$60

Savings: $900/month or 94% cost reduction

EC2/Server Costs#

PHP:

• Needs: 2GB RAM, 2 vCPUs
• Can handle: ~5 concurrent exports
• EC2 t3.small: $15/month

Go:

• Needs: 512MB RAM, 1 vCPU
• Can handle: ~50 concurrent exports
• EC2 t3.micro: $7.50/month

Better performance at half the cost.

When to Use Which?#

Use PHP Version When:#

• You’re already in a Laravel/Symfony project
• Team only knows PHP
• Need tight integration with PHP ecosystem
• Export volume is low (<100K rows typically)

Use Go Version When:#

• Performance is critical (millions of rows)
• Running in containerized environments
• Want minimal memory footprint
• Need concurrent exports
• Deploying to serverless (Lambda, Cloud Run)

Use Both:#

• Laravel app calls Go binary for heavy exports
• Best of both worlds!

Open Source Journey#

Both packages are open source:

PHP Version:

• 📦 kolay-xlsx-stream
• 🎯 For Laravel/PHP developers
• 💚 178MB memory for 4.5M rows

Go Version:

• 📦 kolayxlsxstream
• 🚀 For performance-critical applications
• 💚 2MB memory for 2M rows

Star them on GitHub! ⭐

Technical Deep Dive: Binary ZIP Generation#

The core secret to both versions is understanding XLSX as ZIP files. But Go’s implementation is more efficient:

PHP#

// Needs to build string, then compress
$xmlContent = "<row>...</row>";
$compressed = gzdeflate($xmlContent, 1);
$this->writeToSink($compressed);

Go#

// Streams directly through compressor
deflateWriter, _ := flate.NewWriter(sink, 1)
deflateWriter.Write([]byte("<row>...</row>"))
deflateWriter.Flush()

Difference: Go never holds uncompressed data in memory.

What’s Next?#

Working on:

For Go Version#

• ✅ Styles support (bold headers, number formats)
• ✅ Formula support (SUM, AVERAGE, etc.)
• ⏳ Multiple sheets with different data
• ⏳ Google Sheets streaming
• ⏳ CSV streaming with same API

Integration Examples#

• 📝 Complete Laravel + Go tutorial
• 📝 Docker compose setup
• 📝 Kubernetes deployment guide

The Unexpected Lesson#

I started this journey to solve a PHP problem. I ended up discovering that some problems are better solved in a different language.

But here’s the thing: Both solutions are valuable.

• PHP version: Perfect for Laravel apps, easy to integrate, good enough for most use cases
• Go version: When you need that extra performance, when scale matters, when memory is expensive

The best engineers know when to use the right tool for the job.

Try It Yourself#

PHP Version#

composer require kolay/xlsx-stream

Go Version#

go get github.com/turgutahmet/kolayxlsxstream@v1.0.0

Quick Benchmark#

# Clone the repo
git clone https://github.com/turgutahmet/kolayxlsxstream
cd kolayxlsxstream

# Run benchmark
go run benchmark/comprehensive.go

Watch it export 2 million rows in 3.5 seconds. 🚀

Conclusion#

Rewriting a production-ready package in a different language taught me more than any tutorial ever could:

1. Performance matters — 12x faster isn’t just a number, it’s real cost savings
2. Memory efficiency — O(1) in Go really means O(1)
3. Type safety catches bugs — That Go compiler is annoying until it saves you in production
4. Deployment simplicity — Single binary deployment is magical
5. Right tool for the job — Sometimes you need both

If you’re exporting large datasets, give kolayxlsxstream a try. Whether you choose PHP or Go, you’re getting battle-tested code that handles millions of rows in production.

About the Author#

I’m a software engineer who loves building high-performance tools for real-world problems. Started with PHP, fell in love with Go, still use both daily.

This journey from PHP to Go taught me that the best solution isn’t always about the language — it’s about understanding the problem deeply enough to solve it in any language.

Connect:

• 🐙 GitHub: github.com/turgutahmet
• 🔗 LinkedIn: https://www.linkedin.com/in/turgut-ahmet/

Try the packages:

• PHP: composer require kolay/xlsx-stream
• Go: go get github.com/turgutahmet/kolayxlsxstream@v1.0.0

Star on GitHub:

• PHP: kolay-xlsx-stream
• Go: kolayxlsxstream

Have you rewritten something in a different language? What did you learn? Let me know in the comments!

#Golang #PHP #AWS #S3 #Excel #Performance #OpenSource #Laravel