Hi, this may be overkill for what you're trying to achieve, but if you look at git submodules.
You can nest a git reference inside of another project and manage the referenced repository separately.
Sourcecode/
PlayerModels
EnemyModels
GameController
Resources
Configuration/
where everything is your project. Then you could add an Assets folder under Resources, linking it to my.git.hostingsite:assets.git
And manage the assets project separately - or perhaps by different team members.
What you'll do in your main project is git submodule update as needed to keep your git reference up todate. However, only the *reference* in name, git url and commit hash will be committed to your primary repository.
This is often used when you're using a third-party library that's evolving, or a core game engine across multiple projects. It keeps you from having to duplicate code (or continually paste your Assets in).
Again this may not be an exact solution for what you're looking for but it sounds like you have a clear deliniation between your source code and the Art & DLLs so I recommend looking into it.
https://git-scm.com/book/en/v2/Git-Tools-Submodules