Datatables Scrolling Data not being Called

Hello!

I've been trying to use the datatables scroller to load data dynamically as the scrolling happens. The only problem I'm having is with triggering the ajax to fire again. It doesn't call the ajax in an appropriate position in the table. The same problem occurs in the 50k example on this website.

I've been able to get it to fire by manually setting a height between 25 and 50 viewport height. Here's my initialization:

DOM:

`<table id="permissionsTableList" class="hbs-grid hbs-list" style=" width:98%;height: 25vh !important">
        <thead>

        </thead>
        <tbody></tbody>
    </table>
`

Datatables initialization:

 $('#permissionsTableList').dataTable({
                destroy: reinit,
                "bScrollInfinite": true,
                "bScrollCollapse": true,
                "order": [0, 'asc'],
               "deferRender": true,
                "scrollY": "50vh",
                "serverSide": true,
                "scroller": {
                    loadingIndicator: true,
                    
                },
                "ajax": function (data, callback, settings) {

                    $.ajax({
                        type: "GET",
                        url: "URL?handler=GetData",
                        data: { start: data.start, length: 50 },
                        beforeSend: function (xhr) {
                            xhr.setRequestHeader("XSRF-TOKEN",
                                $('input:hidden[name="__RequestVerificationToken"]').val());
                        },
                        success: function (d) {
                                setTimeout(function () {
                                    callback({
                                    draw: data.draw,
                                    data: d,
                                    recordsTotal: @Model.Data.Count(),
                                    recordsFiltered: @Model.Data.Count(),
                                });
                            }, 50)
                        }
                    })
                },

Any help figuring out this problem is apprfiated.