Question about flash memory as used in an SSD or MCU

Log in or register to post comments
Go To Last Post
15 posts / 0 new
Author
Message
Krupski
Krupski's picture
Level: Hangaround
Joined: Tue. Nov 17, 2015
Posts: 243 View posts
Location: NY, USA
#1
Posted by Krupski: Fri. Dec 13, 2019 - 07:31 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

I hope this is in the correct forum. If not, mods please move it and my apologies.

OK... we know that EEPROM and FLASH memory must be erased prior to writing (that is, bytes or blocks need to first be erased so that all bits are 1, resulting in lots of 0xFF data).

But, a new SSD if not formatted is filled with 0x00 data (i.e. reading it returns all zeroes).

So, my question(s) are:

* Is the data which is read or written INVERTED (that is, would a byte of data 0x55 be stored in flash as 0xAA)?

* If NO, then shouldn't an "erased" drive be filled with 0xFF instead of 0x00?

* Since an erased MCU such as an AVR reads back as all 0xFF, can I assume that data inversion is NOT used here?

(by the way, my "goal" with this is to write a "program more zeroes" algorithm to make flash and/or eeprom cells last a bit longer). It's nothing "important", just a learning experiment.

Any info about this will be appreciated.

Gentlemen may prefer Blondes, but Real Men prefer Redheads!

Tags:

Learning and Information, General Electronics
Last Edited: Fri. Dec 13, 2019 - 07:32 AM
Top
Kartman
Kartman's picture
Level: 10k+ Postman
Joined: Thu. Dec 30, 2004
Posts: 24978 View posts
Location: Melbourne,Australia
Posted by Kartman: Fri. Dec 13, 2019 - 07:46 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

With a ssd, formatted does not mean erased. I have not actually tried reading a raw sector from a ssd, so i have no idea what the initial state is, however, something must be written to initialise the ECC for each sector.

Last Edited: Fri. Dec 13, 2019 - 07:47 AM
Top
Kartman
Kartman's picture
Level: 10k+ Postman
Joined: Thu. Dec 30, 2004
Posts: 24978 View posts
Location: Melbourne,Australia
Posted by Kartman: Fri. Dec 13, 2019 - 07:51 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 1

Also note that ssd uses NAND flash whereas the AVR uses NOR flash. These are very different.

Top
Bronze Markawneil
awneil's picture
Level: 10k+ Postman
Joined: Fri. Jul 1, 2005
Posts: 25563 View posts
Location: Basingstoke, Hampshire, UK
Posted by awneil: Fri. Dec 13, 2019 - 09:04 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Kartman wrote:
AVR uses NOR flash

I think that is true of microcontrollers in general?

 

https://electronics.stackexchange.com/questions/303899/what-type-of-flash-nand-or-nor-is-used-on-microcontoller

 

https://searchstorage.techtarget.com/definition/NOR-flash-memory

 

 

Top Tips:

  1. How to properly post source code - see: https://www.avrfreaks.net/comment... - also how to properly include images/pictures
  2. "Garbage" characters on a serial terminal are (almost?) invariably due to wrong baud rate - see: https://learn.sparkfun.com/tutorials/serial-communication
  3. Wrong baud rate is usually due to not running at the speed you thought; check by blinking a LED to see if you get the speed you expected
  4. Difference between a crystal, and a crystal oscillator: https://www.avrfreaks.net/comment...
  5. When your question is resolved, mark the solution: https://www.avrfreaks.net/comment...
  6. Beginner's "Getting Started" tips: https://www.avrfreaks.net/comment...
Top
Krupski
Krupski's picture
Level: Hangaround
Joined: Tue. Nov 17, 2015
Posts: 243 View posts
Location: NY, USA
Posted by Krupski: Fri. Dec 13, 2019 - 10:54 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Kartman wrote:
With a ssd, formatted does not mean erased. I have not actually tried reading a raw sector from a ssd, so i have no idea what the initial state is, however, something must be written to initialise the ECC for each sector.

I know that of course. What I was saying was that if I buy a new SSD (msata, sata, whatever) it usually comes as blank and unformatted requiring me to first run parted to create the "partition" (the 512 byte thing in sector 0 that has the pre-boot code, the drive geometry, etc... and ending in 0x55, 0xAA). Then I run mkfs.ext4/.ntfs/.exfat/.whatever to format the drive. But BEFORE any of this is done, the drive is blank, unpartitioned and unformatted and every byte is 0x00. This is determined by running hexedit /dev/sdx and looking at the raw data. Any ECC data the drive has is between the storage media itself and the drive controller silicon. That data is not normally exposed to the user and special ATA "read sector raw" commands are required to see that data at all. The error correction codes are not what I was talking about anyway.

Gentlemen may prefer Blondes, but Real Men prefer Redheads!

Top
Krupski
Krupski's picture
Level: Hangaround
Joined: Tue. Nov 17, 2015
Posts: 243 View posts
Location: NY, USA
Posted by Krupski: Fri. Dec 13, 2019 - 11:03 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Kartman wrote:
Also note that ssd uses NAND flash whereas the AVR uses NOR flash. These are very different.

I know that. What I was wondering was if certain controllers in devices like SSDs invert the data so that "1" bits on the data bus are actually stored as programmed "0" bits. I ask because an ERASED flash byte is all 1s (resulting in 0xFF in flash which appears to the outside world as 0x00, leading me to assume that data on the bus is inverted before writing it to flash and inverted back to normal when reading from flash to send to the data bus. You see what I'm asking?

Gentlemen may prefer Blondes, but Real Men prefer Redheads!

Top
Krupski
Krupski's picture
Level: Hangaround
Joined: Tue. Nov 17, 2015
Posts: 243 View posts
Location: NY, USA
Posted by Krupski: Fri. Dec 13, 2019 - 11:20 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Kartman wrote:
...however, something must be written to initialise the ECC for each sector.

An SSD is initialized at the "factory". This includes calculating then writing the initial error correction code data as well as mapping out defective pages of flash (wafer level defects on the chip) and setting up the pool of spare pages of flash (provisioning). During the life of the SSD, ECC data is calculated on the fly and written to flash along with the data itself as well as using the ECC to verify and, if necessary (and possible) correct bit errors in the data read from flash. Also, data is moved around to different flash pages (wear levelling) and pages that die of old age (wear out or have ECC uncorrectable data errors) are mapped out and replaced with pages from the pool of spares. Of course, all of this has nothing to do with what I originally asked in post #1.

Gentlemen may prefer Blondes, but Real Men prefer Redheads!

Last Edited: Fri. Dec 13, 2019 - 11:22 AM
Top
Bronze Markawneil
awneil's picture
Level: 10k+ Postman
Joined: Fri. Jul 1, 2005
Posts: 25563 View posts
Location: Basingstoke, Hampshire, UK
Posted by awneil: Fri. Dec 13, 2019 - 11:27 AM (Reply to #7)
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

So maybe the process of "initialising" at the "factory" leaves the data at all zeros .. ?

 

Krupski wrote:
BEFORE any of this is done, the drive is blank, unpartitioned and unformatted and every byte is 0x00. 

On what do you base this?

 

Is this guaranteed, or is it just what you happen to have noticed?

 

Top Tips:

  1. How to properly post source code - see: https://www.avrfreaks.net/comment... - also how to properly include images/pictures
  2. "Garbage" characters on a serial terminal are (almost?) invariably due to wrong baud rate - see: https://learn.sparkfun.com/tutorials/serial-communication
  3. Wrong baud rate is usually due to not running at the speed you thought; check by blinking a LED to see if you get the speed you expected
  4. Difference between a crystal, and a crystal oscillator: https://www.avrfreaks.net/comment...
  5. When your question is resolved, mark the solution: https://www.avrfreaks.net/comment...
  6. Beginner's "Getting Started" tips: https://www.avrfreaks.net/comment...
Top
Gold Markclawson
clawson's picture
Level: Moderator
Joined: Mon. Jul 18, 2005
Posts: 104758 View posts
Location: (using avr-gcc in) Finchingfield, Essex, England
Posted by clawson: Fri. Dec 13, 2019 - 11:40 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

awneil wrote:

On what do you base this?

 

Is this guaranteed, or is it just what you happen to have noticed?

It's my experience too. As far as I know all of CF Cards, USB memory sticks and SSDs all come with all sectors written to 0x00. As Krupski says this is counter-intuitive as the electrons on the gates of NAND default to '1' in the erased state (so 0xFFs). It does make you wonder if they are doing something "clever" inside to invert the data or something (so that when we read 0x00 the array is really holding 0xFF) but I think only an engineer involved in the design of SSD/CF/MSD would really know what they are doing "inside" the controller software of the memory device.

Top
Krupski
Krupski's picture
Level: Hangaround
Joined: Tue. Nov 17, 2015
Posts: 243 View posts
Location: NY, USA
Posted by Krupski: Fri. Dec 13, 2019 - 11:57 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

awneil wrote:

So maybe the process of "initialising" at the "factory" leaves the data at all zeros .. ?

 

Krupski wrote:
BEFORE any of this is done, the drive is blank, unpartitioned and unformatted and every byte is 0x00. 

On what do you base this?

 

Is this guaranteed, or is it just what you happen to have noticed?

 

I'll amend the statement: All the bytes I can see (i.e. user data, not specialty data such as ECC and defect management) are 0x00.

Gentlemen may prefer Blondes, but Real Men prefer Redheads!

Top
Bronze Markawneil
awneil's picture
Level: 10k+ Postman
Joined: Fri. Jul 1, 2005
Posts: 25563 View posts
Location: Basingstoke, Hampshire, UK
Posted by awneil: Fri. Dec 13, 2019 - 12:05 PM (Reply to #9)
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

clawson wrote:
 It does make you wonder if they are doing something "clever" inside to invert the data or something

Or it it's just part of the "initialisation" process which leaves everything at zero - whether by design or "coincidence".

 

 I think only an engineer involved in the design of SSD/CF/MSD would really know what they are doing "inside" the controller software of the memory device.

Indeed. And whether it is something you can rely upon or not ...

Top Tips:

  1. How to properly post source code - see: https://www.avrfreaks.net/comment... - also how to properly include images/pictures
  2. "Garbage" characters on a serial terminal are (almost?) invariably due to wrong baud rate - see: https://learn.sparkfun.com/tutorials/serial-communication
  3. Wrong baud rate is usually due to not running at the speed you thought; check by blinking a LED to see if you get the speed you expected
  4. Difference between a crystal, and a crystal oscillator: https://www.avrfreaks.net/comment...
  5. When your question is resolved, mark the solution: https://www.avrfreaks.net/comment...
  6. Beginner's "Getting Started" tips: https://www.avrfreaks.net/comment...
Last Edited: Fri. Dec 13, 2019 - 12:06 PM
Top
joeymorin
joeymorin's picture
Level: 10k+ Postman
Joined: Tue. Jul 17, 2012
Posts: 12051 View posts
Location: Location, Location
Posted by joeymorin: Fri. Dec 13, 2019 - 03:58 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 1

clawson wrote:

It does make you wonder if they are doing something "clever" inside to invert the data or something

Or something.

 

All modern SSD use MLC.  Notions of 0 and 1 per cell are no longer accurate.

 

Krupski wrote:

I'll amend the statement: All the bytes I can see (i.e. user data, not specialty data such as ECC and defect management) are 0x00.

You're seeing what the SSD controller is showing you.  You do not have access to the flash cells themselves.  For all you or I know, an all-zero 'device' sector is handled specially.  Maybe it's just a flag that says to the controller "ignore the contents of this sector and just report all zeros".  This would make sense to me since an all-zero sector is common with typical filesystem usage.  Special handling would reduce wear in these cases.

 

You can't draw much of a meaningful comparison between and SSD with GB of high-speed storage, and a microcontroller with a few kB of single-port flash.

"Experience is what enables you to recognise a mistake the second time you make it."

"Good judgement comes from experience.  Experience comes from bad judgement."

"Wisdom is always wont to arrive late, and to be a little approximate on first possession."

"When you hear hoofbeats, think horses, not unicorns."

"Fast.  Cheap.  Good.  Pick two."

"We see a lot of arses on handlebars around here." - [J Ekdahl]

 

Top
N.Winterbottom
N.Winterbottom's picture
Level: Raving Lunatic
Joined: Tue. Oct 1, 2002
Posts: 2809 View posts
Location: Redbourn, Herts, UK
Posted by N.Winterbottom: Thu. May 12, 2022 - 09:42 AM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Krupski wrote:
But, a new SSD if not formatted is filled with 0x00 data (i.e. reading it returns all zeroes).

You already acknowledge that this sector read is performed via a controller, but this controller is a complex afar.

 

We know some mapping of logical sectors to physical sectors must exist for bad sector management and wear levelling purposes. I'm guessing there is some EEPROM on the controller for housekeeping.

The controller also knows which blocks of physical sectors are erased or pre-erased.

Therefore when a read for a sector in an erased block is requested, the controller doesn't even have to read the physical sector. It can simply return all zeros.

 

 

Top
barnacle
barnacle's picture
Level: Raving Lunatic
Joined: Tue. Jan 3, 2006
Posts: 7291 View posts
Location: Berlin DE
Posted by barnacle: Thu. May 12, 2022 - 04:56 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

Um, is this a zombie thread resurrection complete with spam?

 

Neil

Neil Barnes
www.nailed-barnacle.co.uk

Top
Gold Markclawson
clawson's picture
Level: Moderator
Joined: Mon. Jul 18, 2005
Posts: 104758 View posts
Location: (using avr-gcc in) Finchingfield, Essex, England
Posted by clawson: Thu. May 12, 2022 - 05:05 PM
  • 1
  • 2
  • 3
  • 4
  • 5
Total votes: 0

HAD spam. Sadly his post, his account and his very being seem to have left us :-(

 

Feel free to add anything you think relevant but, as Neil points out, you are responding to a zombie ;-)

Top