so what is the worst thing that will eventually happen to these "artifacts"
1. Depending on the nature of the object, the silicone adhesive can etch into it. It might be very slight, but it can be enough to damage and discolor the surface. Think things like old silverware, medals, coins, ceramic.
2. When the object is later removed, it's pretty rare that the silicone just peels off like everyone seems to think it will. It always takes some effort at scraping it off, which could easily damage the object.
3. If the object is porous the silicone will permeate into it. Even if you can scrape it all off the surface, it will still be in the object, probably never to be removed.
4. If it's flakey, for lack of a better word, the adhesion of the silicone can pull layers of it off, either when it's removed from the shadowbox or if it's bumped too hard.
5. Finally, and this is true of all adhesives, you're relying on the strength of the top layer of the mat. Usually if an object falls in a shadowbox you'll find that the glue held, but everything peeled away from the mat.
Just a few things to consider, I'm not the Framing Police or the ultimate authority by any means.