Now, I'm not an expert in any of the fields you mention. So let me tread carefully.
- Category stuff is certainly a state-of-the-art slant on mathematics, and there is at least one undergraduate math book using categories from the outset [REF]. To what extent this is the most suitable way to get to know the width and depth of mathematics is not for me to say. You can also do most of math without delving into the formalities and anive set theory will get yo a long way.
- There are many ways you can "do" formal systems, of which set theory is a specialization. And indeed, there is more than one axiomatic road into set territory. Which one you take may depend on your preference or objectives. One may also regard these alternatives as a topic for comparison an study in itself, taking a meta-theoretical perspective.
- I will not attempt to give Yet Another Definition of syntax. In a broad sense, syntax is about rules for the proper construction of sentences, I believe. This has a descriptive side to it ("aha: look, this is how a sentence is constructed") as well as a prescriptive one ("a sentence must be constructed like this"). And it seems that "syntax" has to do with languages, either natural or formal, being the habitat or even raison d'être for sentences...?
- Now if I have a set of such rules, and a vocabulary, I can construct proper ("well formed") sentences (possibly ignoring some other aspects of grammar now). Or, presented with a candidate-sentence, I may use the rules to find out whether it is OK (= complies to the syntax). If this is all there is to language, this is likely to become boring soon.
- Indeed, that any such sentences may also convey something ("meaning") beyond their own shape, appearance and syntactical correctness, is a question to which syntax is oblivious. The other way around, syntax rules are probably a necessary evil for any (set of) sentence(s) to convey anything at all in a systematic and reproducible way.
- Could we say that this "something more" is somehow a relationship between a given (say, syntactically proper) sentence (or text made up of sseveral such), and something else. This something else may itself reside in another linguistic or formal system (e.g. set theory in some guise), or in, well... our natural environment. This rapport or reference may be the purpose of the language (and its syntax) to begin with. I presume this relationship has to specified implicitly or explicitly, by decree or by convention. All anive and primitive maybe, but this would be my initial understanding of semantics in a very general sense.
- So in as far as we can agree on the above outline, syntax is a suitable convention about sentences that are formally properly constructed. Not being too subtle about it, syntactically correct sentences are not by themselves meaningful (or full of meaning). At best, they are the (sole?) candidates to bear / be endowed with meaning. But adding this requires something extra. This additional convention for making (or agreeing to make) sentences refer to something else, or the study of such conventions, to me would constitute "semantics".
- One way to insert meaning is to start at an atomic level, the one of the vocabulary.