Working with Lazy Collection in Laravel 6.0

Laravel 6.0 introduced Lazy Collection. Lazy collection leverages PHP’s Generators to allow you to work with a very large dataset and keeps the memory usages low. Let’s take an example with 60,000 users records.

Now, what if we try to access all the users eloquent model at once? Yes, we will get 500 error due to memory overload. Because of all the 60,000 users first, have to load into the memory.

$users = \App\User::all();

To get rid of this memory usage we can use the cursor() method. This method will allow you to iterate through your dataset records and will only execute a single query. This method will return a Lazy Collection and allows you to use the existing collection methods. In this case, no query is being generated.

Route::get('/', function() {

    $users = \App\User::cursor();
    dd($users);
});

Now if we can apply the collection methods to the Lazy Collection, let’s call the first() method.

$users = \App\User::cursor();
dd($users->first());

Now a single query is being generated and return the single record. Let’s implement a filter() collection with Lazy Collection.

$users = App\User::cursor()->filter(function ($user) {
    return $user->id > 500;
});

foreach ($users as $user) {
    echo $user->id;
}

Through this we don’t face any memory issues because of lazy collection. Take a look at how’s the cursor method works.

/**
 * Get a lazy collection for the given query.
 *
 * @return \Illuminate\Support\LazyCollection
 */
public function cursor()
{
    if (is_null($this->columns)) {
        $this->columns = ['*'];
    }

    return new LazyCollection(function () {
        yield from $this->connection->cursor(
            $this->toSql(), $this->getBindings(), ! $this->useWritePdo
        );
    });
}

Read Large File

Alternatively, we can use the lazy collection to read huge gigabytes of file. Let’s take an example to read a huge log file. We are going to use the LazyCollection class to achieve this. The make() method will help to chain the entire section with other helper collections. In this example, we take 4 records with chunk() method and load the records to the LogEntry model and iterate through that records.

use App\LogEntry;
use Illuminate\Support\LazyCollection;

LazyCollection::make(function () {
    $handle = fopen('log.txt', 'r');

    while (($line = fgets($handle)) !== false) {
        yield $line;
    }
})
->chunk(4)
->map(function ($lines) {
    return LogEntry::fromLines($lines);
})
->each(function (LogEntry $logEntry) {
    // Process the log entry...
});

As you noticed the yield keyword. The yield keyword is similar to the return statement, but instead of stopping the execution of the function and returning, yield instead provides a value to the code looping over the generator and pauses execution of the generator function.

Read Large File

Related posts: