Materials: Much Ado About Nothing

"...They say materials are fair. ‘Tis a truth, I can bear them witness. And good to have—’tis so, I cannot reprove it. And useful, but for SL. By my troth, it is no addition to their function—nor no great argument of the lack of it, for we will be horribly in love with them."
                                                      -Shakespeare, kinda

By now the majority of us have become aware that materials - normal maps and specular maps - will become available as part of the main client soon and used inside SL. To elaborate on the specifics of what they are and how they work I will leave up to the plethora of sources already documenting those aspects. The goal of this post is to focus more on the more immaterial (pardon the pun) aspects of the application of this technology into our virtual lives and some things to consider about how they will most likely impact them.

Obviously, the fashion industry will, as with most new features introduced in the last 3 or 4 years, be at the forefront of taking advantage of it and pushing it forward into the spotlight of mass-adoption by the residents and other markets. This is a good thing and one of the wonderful consequences of such a dominant element in SL. You will most likely see it first on some article of clothing well ahead of ever seeing it on a house, chair or plant.

In the maelstrom of excitement about the advent of this fancy new shiny (literally) that the lab hath bestowed upon us, it is important to understand a few things about it that are not generally being spoken about but are worth considering. I am no voice of reason, but cannot help but notice that in all the confusion a few important aspects are being glossed over as sure as any specular map.

Yes, they will improve the look of things, or at least make things look different than they do now, much like sculpties and mesh did in their time. We will look back sometime in the future from an SL which looks perhaps vastly different than it does now. Things will be bumpier and shinier, if not more realistic.

As this is implemented, lots of people will be trying these features and releasing lots of new items that utilize the new maps. In the process, as with sculpties and particularly with mesh, much of it will be done simply for the sake of doing so. Much of it will also be done in an attempt to make things better, or look better. This is as it should be. In the mad rush to take advantage of what normal and specular maps can do, however, there will also be a considerable amount of overuse and misuse of them. Shiny for shiny's sake, if you will.

Simply look to the past implementation of a variety of features in SL and you see it happen again and again. Part of me knows that is just part of the process but part of me also hopes that it doesn't happen as much as I think its going to. Mesh items with ridiculously high polygon counts are still being produced in crazy numbers. LOD meshes and accurate rigging are still the practice of a small handful of the small handful of designers producing mesh on the grid.

There is a right way to go about it and a wrong way to go about it, both in how materials will be used by creators and by how the feature is being implemented by the lab. Unfortunately, there are already flaws in the system which have nothing to do with bugs in the code. Consider that normal and specular mapping are proven and established technologies that have existed outside of SL for over a decade. Many examples of how they can be created and how they can be used exist.

To be clear (and this is critical to understand):

Normal and Specular mapping was implemented as a way to render low polygon objects inside an environment in such a way that they appear to have much more surface detail than they actually do, in order to enhance the visual while maintaining efficiency of the models.
_______________________________________

The ideal process for using them goes something like this: A low polygon model is created. The model is then sub-divided (the shape stays roughly the same but is divided into many many more polygons). The low-poly model is kept. Very high levels of surface detail can then be added or sculpted using that high-polygon model, i.e. wrinkles, skin, folds, buttons, seams, cracks, etc. Once the model is finished in high detail a normal and specular map (texture) are created which save all those details. These maps are then applied to the low-polygon item inside your game environment. The rendering engine (in SL its windlight) then interprets the normals and specular details from those maps and renders them accordingly, making the low-poly item appear as if it has much more detail than it actually does. It is a cheat. A very clever, well designed cheat made to fool the eye into seeing lots of surface geometry that does not really exist on the actual model as anything other than a texture map. This is the most difficult method for creating and using normal maps, and will be the method which is, as a result, the least used in SL (at least initially).

There are, however, additional methods for using and creating materials which will be much more prevalent in SL. They can be generated based on an existing texture, which can be done almost instantly and efficiently by programs like CrazyBump and even Photoshop, producing a usable and decent addition to the detail of that existing texture in ways that enhance the appearance in world. Wood grain, bricks, stone walls are good examples. Materials can also be created as an all over, repeatable pattern which has little reference to the image texture used on an object and in fact can be completely different from that image. The water in SL, velvet or suede on a jacket are good examples of this method. They do not represent the texture or add specific detail to an object but do enhance the rendering of it.

Unfortunately material maps will be used in SL to add even more detail to already high polygon models, which is fantastic if you are rendering a movie scene or a high-detail still image but can be horrible for a game environment and even more so in a live virtual environment like SL. Take a look at the models in just about any popular game out there. None of them have on armor or corsets or hair or shoes that have 40k-50K polygons. Yet in SL, this is common. Less so on furniture and houses, as the land impact becomes a factor, but still prevalent on smaller items. Consider a sim crowded with avatars loaded down with high polygon mesh and surrounded by mesh items, like any of the recent large events or even the smaller monthly events. Performance is at a standstill. That world of permanent grey isn't going to render any faster when you add additional texture resources to that picture. That is a problem.

To be fair, LL cannot be fully blamed for introducing materials any more than they should be blamed for introducing mesh. BOTH are attempts to provide methods for us to build a better looking world that runs more efficiently than it did before. Ideally. They can, and should be blamed, however, for introducing those "features" in a way which only provides one at the sacrifice of the other. As we see with 1024 textures on nano-earrings and essentially unlimited polygon counts on attachments, the freedom to choose wisely is often abused in the race to provide better, more realistic items for the consumer who has been convinced to demand them. Prim equivalent limiting the use of rezzed mesh items but not attachments is something which should be addressed. They both, arguably, have an impact on the performance of both the server and the client.

For certain, a vast number of SL residents - perhaps even a majority - will not be able to even see materials. In order to do so "advanced lighting" must be turned on in your preferences, which is where enabling shadows also exists, something which requires a certain if not considerable amount of video rendering capability. Running it on even a powerful laptop is not something I would recommend. There are still people out there who cannot see mesh yet. All this being said,  the advantages, beyond the obvious visual ones, that this will provide for us seem rather insignificant.

Another aspect of the way that materials are being implemented which is ineffective concerns the lack of ability to adjust them once uploaded. Specular mapping has a variety of very cool ways in which it can be adjusted, particularly in regards to how it can also work with transparency. Normal maps do not. Once applied, they are on or off. This is problematic, though it will result in either a lot of temp uploads or a lot of upload fees for SL, neither of which is efficient. Let me explain why.

In the process of creating material maps, one of the key aspects is understanding and being able to see how they will work within a particular rendering engine under varying conditions. Granted, a specific amount of resulting quality can be expected when created properly. When uploaded and applied to a model in an environment, however, the results can often be different than expected. Different rendering engines produce very different results. What will work in one will often not work as well in another and must be adjusted and tweaked. In fact, the final results may often depend on that ability to tweak inside the environment. While working inside both Bluemars, which used the CryEngine, and now Cloud Party, which uses OpenGL, the specific look of materials on items I made often came about simply because I spent time messing with the settings, not something I could have predetermined. The varying windlight settings inside the many different regions of SL further compound the problem, something which generally does not happen in most video games, though the lighting may change.

This problem facilitates the need to make adjustments, the ability to increase or decrease the level or the specular and normal maps in order to get the desired result. When trying to create something like suede or velvet this is much more of a problem, as opposed to something much less subtle like leather or glass. On or off is not really ideal in terms of workflow for the creator, as it would require considerable uploading, editing and re-uploading of items in order to produce the desired result. Messing with things is not only critically important, its a fun part of the creative process, now removed. Why?

Perhaps the biggest argument during the process of implementing this new feature, as problematic and short-sighted as it is being done, goes even beyond the scope of those problems, which are not inherent in the technology but are inherent in the company which is producing them. In handing us this bright new shiny, which was not even generated by anyone originally employed by the lab but from someone outside of it because they fired pretty much anyone with considerable talent, I am left feeling like the time and money could have been better spent.

As much as materials are long overdue in SL, as was mesh, and as much as they will make a considerable impact on the way the grid looks and we inside of it look, let me put forth the proposition that we should not be so distracted by this bright new shiny that they are dangling before us ("look what we did for you!") and consider that we should be asking, demanding more from them. There are so many ways in which our world and our imagination could be improved, ways which we have been damn near screaming and begging for. Those features have been ignored and placed on the back burner (sometimes for YEARS) for whichever fancy-du-jour makes it onto the to-do list hanging on the wall in the lab development cubicle. We should expect more.

• We should have the mesh deformer.

For Christ's sake, we PAID for it, developed it and handed it to them over a year ago. Gratis. Working. What message are we to interpret from continued delay on this becoming a real, working feature? Like it or leave. That is the message we get over and over from our benevolent dictators. I don't think that is enough, and I suggest that you shouldn't either. It is not a matter of deserving better, we pay for better and don't get it far too often.

• We should have a new avatar.

The one we have, have had for 10 years, was not that good to begin with. It is now showing it's age like an old pickup truck. It would take no more effort to introduce a new one than it has to introduce materials in SL, perhaps even less. Yet the lab continues to change the oil and kick the tires, wash the windows and let it keep on driving down the road.

Part of the difficulty in introducing a new one is that doing so would most likely involve not only a new body mesh with new UV and geometry, but a new skeleton with new joints as well. This would break a huge amount of existing "legacy" content, from system clothing to AO's to animations and everything that uses them. The only way to create a new avatar effectively then would be to provide both, with the option for the user to select one or the other at will. A choice to have old or new depending on nothing more than your desire.

This can be done, should be done and is the method which has been suggested by a large part of the community which is already involved with development of the mesh deformer, oddly enough. There is a reason for that connection.

The mesh deformer is dependent on the geometry of the existing avatar in oder to deform. Mesh clothing is generally very well made and often has a higher polygon count than the underlying body on which it is worn. The flaws in the existing avatar become very apparent when you put a really nice mesh blouse over it and deform it. In order to have the mesh deformer, one that works really well and properly, it is essential to have a new avatar on which to wear it. This is a primary reason we don't have this feature yet. Nobody at the lab wants to light the fuse on the bomb of angry response if everyone's clothing suddenly gets screwed up by the crappy avatars, and they also don't want to light the fuse on the bomb of angry response that will happen if everyone's AO, animations, sex beds and rigged clothing suddenly does not work either. It is a catch-22 for them.

SO, in order to see a mesh deformer we are going to have to first see a new avatar, and the choice to pick the new or old one. That is a lot to expect from the lab and I have doubts that they can deliver something so advanced and so critical to the continued success of SL. The track record they have in doing the right thing at the right time in the right way does not bespeak of a favorable outcome.

Understandably this article will go against the grain of acceptance for many in the community. Please do not misunderstand the points made, or my goal in writing this. I do not hate SL, far from it. Though I have explored (will explore) other worlds I continue to come back to it and the community and friends which are there. Working deformer or no, materials or no, the community is the one single thing which is right about SL. Oddly, its also the one thing the lab has never had anything to do with. There is no love lost for the company which runs SL. I have a large callus on my ass from the number of times I have been kicked by them.

As wonderful as materials will be, there are many other ways which the lab could be spending its time and limited resources to improve SL instead of developing, say, new games that have nothing at all to do with SL. Let us hope that they will begin to do so once materials are released, because the message that many of us are getting at this point is that we are no longer a priority. Let us have hope that we will see some features implemented which will benefit all of us, not just some of us. Sooner rather than later.

"Someday, maybe."