It's not your imagination. When AdBlock for Microsoft Edge is running as Edge is opening, you may indeed see some lag. Our 184.108.40.206 update of AdBlock for Microsoft Edge improved things as much as we can, but any significant improvement needs to wait for Microsoft. There are two factors that affect this, neither of which we have any control over.
On top of that, if your computer or your Internet connection is a little bit slow (and especially if both of them are), it takes longer for AdBlock to download your subscribed filter lists on startup. Be sure to subscribe to only the filter lists you absolutely need.
We're all hoping the first two factors get better as Microsoft Edge matures. In the meantime, the lag generally seems to occur only when Edge is opening and AdBlock is loading the filter lists you're subscribed to. After that, browsing should be faster.
An issue that's related to this is that you may see ads while Microsoft Edge is opening. If they go away when you refresh the page, please see this Knowledge Base article.