Can someone explain me the concept of environments?

Yii 2.0 General Discussions
1

I don’t understand the concept of environments. Can someone explain it to me?

In particular:

  1. What kind of settings (groups or examples, please) should be stored in ignored -local files and which can be safely stored in repo (in non ignored files)?

  2. What kind of settings (again, examples, please) should be stored in backend/config, common/config and frontend/config and which one should be set in environments folder?

  3. How often should I init the application into dev or production environment on the same machine. Is it a daily based routine (i.e. after each commit) or rather one-time shot after cloning repository in current version to the brand new environment?

  4. Why *-local.php files for backend/config, common/config and frontend/config are added to .gitignore file, but the very same files in environments/*/*/backend, environments/*/*/common and environments/*/*/fronted folder are not? What is the reason for committing these files to repo?

  5. If I should store all my local configuration in git-ignored *-local.php files (so my passwords won’t end up in repo and won’t overwrite the same settings for others) then what is the reason for storing anything in environments/*/*/backend, environments/*/*/common and environments/*/*/fronted at all?

  6. If I should store all the sensitive configuration in the locally ignored files then what is the benefit or purpose of using environments at all? I install my app on the new environment, init it to i.e. production environment and still I have to manually edit (locally ignored) configuration files). Why then bother at all with all this environments, init stuff etc.?

  7. Why mail transport configuration, password minimum length, support email, admin email etc. are all stored in non-locally-ignored, repo-committed common/config/params.php file? Shouldn’t such configuration be environment specific and therefore stored in a corresponding sub-folder and file in environments folder?

Thank you in advance. Sorry, for bothering, but unfortunately I am getting a huge feeling that the corresponding guide section doesn’t actually explain much.

2

The installation of the environment is done either on each deployment or on first project init. Originally, the concept has emerged from the need to have multiple websites hosted on a single server when OS environment variables might have name conflicts.

Non-secrets could be added to environments and committed. Secrets should not. Either you take these from environment variables (then you put these values to envrionments as well) or writing the values directly to config adjusting -local.php files there.

Backend is something for admin panel. For example, panel-specific routes, config for external APIs used at admin panel only etc.

Frontend is something that faces end user. It makes sense to configure these specific services/components there.

Common is something that’s used in both admin panel and the part facing end-users. For example, DB connection or cache.

Depends on how you deploy your apps. I used it both ways.

Because environments is a “template” which is used by ./init only. The reason for committing these is someone to be able to quickly get almost working configs for a certain environment.

Because these configs contain not only secrets but normal configs as well.

Correct. That is just a template (see the explanation of how it was created at the very start). You can, instead, use OS environment variables if you feel it more comfortable and less complicated.

Because these do not depend on the environment. If they do, they should be moved to environments, indeeed.

2 Likes
3

Samdark, thank you for a wonderful, complex and comprehensive answer.

There are only two things that I still don’t get. Or actually one, because both questions narrows down to one.

Depends on how you deploy your apps. I used it both ways.

Does this mean that in those project where you deploy your apps (and use init) on daily basis you have to manually update i.e. database settings stored in i.e. /common/config/main-local.php? Because this file is overwritten each time you use init command?

Update database configuration (or otherwise app will die – due to the fact that repo / environments folder contains non-sensitive, default values only) on every deploy / use of init command? Sounds like an overkill.

The environments folder contains (by default) *-local.php files. I would rather expect it to contain regular (non-local ones). Or I am still not getting the whole picture here.

I simply don’t get the idea that -local.php files (actual, locally ignored, tend to contain sensitive configuration) are overwritten with the default (non-sensitive, template, repo-commited) ones each time you use init command.

The idea of having locally ignored -local.php files, containing some actual configuration for me is that these files are never overwritten by init command. Or I am not getting the whole picture here.

4

In this case I’ve used getenv('XXX') in environments or included a constants.php file I was getting elsewhere (from Hashicorp Vault).

./init was ment for the fist initialization only initially.

5

./init was ment for the fist initialization only initially.

Now, I am totally lost! :]

If init is meant for first initialization only, but in some project you init the application into dev or production environment on daily basis then how do you achieve that? Is there any other way to switch from DEV to PROD, if init should be used only once (at the project initialization)?

6

Hi @trejder

I also struggled a bit when I begin use the advanced template and environments. But it’s really handy.

First of - you have to realize, that the “init” thing with environments templates and the “-local” files are two different things which “only” overlap in some parts. In other words – “-local” files are always environment dependent but not vice-versa. There are other files which doesn’t contain any secrets but are environment dependent – like index.php and init tool solve these all.

With this info back to your “init” problem.

In many scenarios I would say you only use init when you “create app on new environment”. And this is really important – the more environments, the more use set up these app, the more you have “apps” in you project (like frontend, backend, blog, api, …) the more are environments helpful. So in my case?

  • Production server
  • Production server but testing version
  • Internal testing server
  • My local server is also one of the setup!
  • Tests
  • And mainly this was new for me – any other person who work in the team and runs his own local machine or even wants to run in on server use the power of environments

Multiply it by 7 apps – common, console, frontend, backend, our-api, app-for-other-api1, app-for-other-api2 a it’s a loooot of configs which are set up for me or any other person in team.

Now to the second part of the “init” question which samdark didn’t fully answer I think – sometime you really use init even if the project is already set up. When and how to handle overwriting the files – this is actually a good question of yours and the creators of yii thought of that.

I use init for already set up project whenever some environment configs are modified. So let’s say I changed few of the index.php files in web folders, changed, what files they are loading (which I actually did) and what environment they are set for(which I actually changed – I added new environment “local”).

In these files, there are no secrets, no empty places which you have to fill when set up the project, but they are environment dependent so that’s why they are part of the environments.

So in this case, I change the environment templates, create commit, then distribute it and am able to run init on any place and my index.php files are updated based on the environment they are (or I chose when running init).

But then you ask – “wouldn’t I overwrite my “-local” configs by this?”. Great question. No – you wont overwrite them. How? You have two options.

Dull – when running the init, the process just asks you for every already existing file, which is about to overwrite. So you can easily chose what to overwrite and what to keep. But this can be very frustrating for many files and many runs.

Clever – you have the check the “index.php” in the root of environments folder which the manuals mentions - Additional Topics: Adding more applications | Yii 2 Advanced Project Template | Yii PHP Framework. It’s very important file and it’s like config for the init process. You can config

  • Files which should be skipped if they already exist – this solves the previous problem
  • You chose where the init should generate validation keys – very handy
  • Files which are writable (permission for the server)
  • Files which are executable (permission for the server)

And all those settings can be different for every environment. And it’s really powerful and all those settings are already used in the default index.php. You should definitely check it.
It’s a shame the manual doesn’t say more about it but already is documented in the file.

There is only one scenario which this workflow doesn’t cover – when I want to change one of the “-local” file, for example the app newly uses ftp and needs to add credentials for it, then while running the “init” I’m force to chose to overwrite the file (and then re-add all the info again) or not overwrite (but then manually edit the file). Or I can put these settings to new file, but this isn’t very clean solution.

It’d be great if init could somehow only update the new part of the config. Something like git does. But I guess it’s too complicated. Or clever solution would be use the approach used when generating the validation keys – the init could read the file, store the content, then overwrite the file and re-add the ones which were present in the previous version. Yep, this would be super cool.

But even without it it’s still handy tool.

You also mentioned using init for switching the environments. I actually can’t imagine scenario where you would like to do that. Once the project instance is set up, it serves it’s purposes. Why would you like to for example switch production to develop? I

1 Like
7

Thank you for a very complex answer that adds a lot to samdark’s answer and that explains everything that I still doubt about or had question about.

Your answer generally summarize everything, so allow me to cite only a few small parts out of it. The whole big rest is as clear as the sunny whether. So I don’t need to ask anything about that rest.

Actually, I start from the very bottom:

You are 100% correct! The whole big mess I had in my head and all the questioning of role of environment has come from the fact that I misunderstood the very purpose of this tool. It is a (one-time-in-most-cases) deployment tool while I was thinking about it like an environment switcher tool.

So I was more like thinking: Well, I am done with this or that change, it works just perfect in the DEV environment. Let’s switch my local machine to PROD environment, to see how it goes there.

No, I see that this is a completely wrong approach. There should be a separate “physical” (hardware or cloud, but still separate machines) environments, all of them standing all the time for their key role. So, if I want to test my recent change in PROD, I should deploy it to PRE-PROD or UAT (test environment with a copy of PROD database) and test there rather than switching my local machine to “act like PROD”.

Understanding this clarified a lot of things. Thanks for helping me figure this out.

Are you 100% sure that it works like this? I mean: If I mark some file as “skipped”, it will be skipped only, if already present at the destination? This is extremely important to build up my whole vision of this. Are we 100% sure that, if file mentioned in “skip list” does not exists, it will be created from source (the environments folder)? Or it will be skipped as well?

Does “skip list” mean “skip, if exists”? Or does it mean “skip always”?

With your answer in mind I think, I’ll give another shot to the idea of keeping configuration in -local.php files.

But, to be 100% honest with you, I’d say that samdark’s idea of:

  • Keeping secrets in system environments only
  • Getting these values with get_env in repo-commited config.php etc. files
  • Completely forgetting about confi-local.php etc. files

Sounds much, much more convenient to me.

From the perspective of other team member using my README.md guide and reading there a step:

X. Put database name, login and password into system variables called…

rather than:

X. Create config-local.php files and put database name, login and password there.

have completely no difference. It is nearly the same process.

But the first approach has the great deal over second one that you never risk overwriting the secret data (because it is stored in system environments, not in files). You can use php init as many times as you want (accidentally or not) and everything is still fine.

And the most important – you can them go back to the “environment switched” idea. Because you can very easily disable Yii debug mode. With simple run of php init instead of editing files.

And you can run app in PROD mode on DEV database.

Thank you again for your very complex and extensive answer.

8

Yep, that’s the whole point. In the code the comment says it clearly: “// list of files that should only copied once and skipped if they already exist”.
U use it so it’s tested. I have a list of files I want to skip stored in a variable. Then I use this variable repeatedly for every environment. The same I have for writable files and validation keys.

Are we 100% sure that, if file mentioned in “skip list” does not exists, it will be created from source (the environments folder)? Or it will be skipped as well?

Yes, what would be the point if the files had been skipped every time. In that case they don’t even have to exist, since they are always skipped.

But, to be 100% honest with you, I’d say that samdark’s idea of: …

I didn’t even knew there is such an option. But for me I think it’s no-go, because even I’m able to set system env on local, I’m not able to do it on server, which is probably a scenario of most of the people. But if you can do it and it suits you more, I can imagine it could be proper solution.

rather than:

X. Create config-local.php files and put database name, login and password there.

Well, that’s the whole point - they don’t need to create the config-local.php, the init do it for them. And it does other things, like it crates unique validation keys or set writable files. Those are things which your team have to do on top of the setting the variables into system env.
So from this point of view, the Yii environments solution seems to be a bit more efficient.

But the first approach has the great deal over second one that you never risk overwriting the secret data (because it is stored in system environments, not in files). You can use php init as many times as you want (accidentally or not) and everything is still fine.

That’s true, that’s definitely an advantage, if you combine the Yii environments with system variables. You can use power of both.